You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Siddharth Seth (JIRA)" <ji...@apache.org> on 2011/09/21 02:10:09 UTC
[jira] [Created] (MAPREDUCE-3054) Unable to kill submitted jobs
Unable to kill submitted jobs
-----------------------------
Key: MAPREDUCE-3054
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3054
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.0
Reporter: Siddharth Seth
Found by Philip Su
The "mapred job -kill" command
appears to succeed, but listing the jobs again shows that the job supposedly killed is still there.
{code}
mapred job -list
Total jobs:2
JobId State StartTime UserName Queue Priority SchedulingInfo
job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
mapred job -kill job_1316203984216_0002
Killed job job_1316203984216_0002
mapred job -list
Total jobs:2
JobId State StartTime UserName Queue Priority SchedulingInfo
job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
{code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3054) Unable to kill submitted jobs
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13116436#comment-13116436 ]
Hudson commented on MAPREDUCE-3054:
-----------------------------------
Integrated in Hadoop-Mapreduce-0.23-Build #30 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/30/])
MAPREDUCE-3054. Unable to kill submitted jobs. (mahadev) - Merging r1176600 from trunk
mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176604
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientCache.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ResourceMgrDelegate.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientRedirect.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestYARNRunner.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ClientRMProtocol.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/FinishApplicationRequest.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/FinishApplicationResponse.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/KillApplicationRequest.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/KillApplicationResponse.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/FinishApplicationRequestPBImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/FinishApplicationResponsePBImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/KillApplicationRequestPBImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/KillApplicationResponsePBImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/client_RM_protocol.proto
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_service_protos.proto
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/client/ClientRMProtocolPBClientImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/service/ClientRMProtocolPBServiceImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/TestRPC.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/resourcetracker/InlineDispatcher.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/TestRMAppTransitions.java
> Unable to kill submitted jobs
> -----------------------------
>
> Key: MAPREDUCE-3054
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3054
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch
>
>
> Found by Philip Su
> The "mapred job -kill" command
> appears to succeed, but listing the jobs again shows that the job supposedly killed is still there.
> {code}
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> mapred job -kill job_1316203984216_0002
> Killed job job_1316203984216_0002
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3054) Unable to kill submitted jobs
Posted by "Arun C Murthy (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-3054:
-------------------------------------
Status: Open (was: Patch Available)
javadoc warning
> Unable to kill submitted jobs
> -----------------------------
>
> Key: MAPREDUCE-3054
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3054
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch
>
>
> Found by Philip Su
> The "mapred job -kill" command
> appears to succeed, but listing the jobs again shows that the job supposedly killed is still there.
> {code}
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> mapred job -kill job_1316203984216_0002
> Killed job job_1316203984216_0002
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3054) Unable to kill submitted jobs
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated MAPREDUCE-3054:
-------------------------------------
Attachment: MAPREDUCE-3054.patch
Added more units tests for the client side plugin. Also, added tests to RM app transitions to make sure they get killed.
> Unable to kill submitted jobs
> -----------------------------
>
> Key: MAPREDUCE-3054
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3054
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch
>
>
> Found by Philip Su
> The "mapred job -kill" command
> appears to succeed, but listing the jobs again shows that the job supposedly killed is still there.
> {code}
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> mapred job -kill job_1316203984216_0002
> Killed job job_1316203984216_0002
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3054) Unable to kill submitted jobs
Posted by "Mahadev konar (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated MAPREDUCE-3054:
-------------------------------------
Attachment: MAPREDUCE-3054.patch
Fixed javadoc warnings.
> Unable to kill submitted jobs
> -----------------------------
>
> Key: MAPREDUCE-3054
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3054
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch
>
>
> Found by Philip Su
> The "mapred job -kill" command
> appears to succeed, but listing the jobs again shows that the job supposedly killed is still there.
> {code}
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> mapred job -kill job_1316203984216_0002
> Killed job job_1316203984216_0002
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3054) Unable to kill submitted jobs
Posted by "Mahadev konar (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated MAPREDUCE-3054:
-------------------------------------
Status: Patch Available (was: Open)
> Unable to kill submitted jobs
> -----------------------------
>
> Key: MAPREDUCE-3054
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3054
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch
>
>
> Found by Philip Su
> The "mapred job -kill" command
> appears to succeed, but listing the jobs again shows that the job supposedly killed is still there.
> {code}
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> mapred job -kill job_1316203984216_0002
> Killed job job_1316203984216_0002
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3054) Unable to kill submitted jobs
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated MAPREDUCE-3054:
-------------------------------------
Attachment: MAPREDUCE-3054.patch
Update patch which fixes the transitions in RMApp to make sure RMAttempt is also killed.
Still pending testing and unit tests.
> Unable to kill submitted jobs
> -----------------------------
>
> Key: MAPREDUCE-3054
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3054
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3054.patch, MAPREDUCE-3054.patch
>
>
> Found by Philip Su
> The "mapred job -kill" command
> appears to succeed, but listing the jobs again shows that the job supposedly killed is still there.
> {code}
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> mapred job -kill job_1316203984216_0002
> Killed job job_1316203984216_0002
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3054) Unable to kill submitted jobs
Posted by "Mahadev konar (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13116138#comment-13116138 ]
Mahadev konar commented on MAPREDUCE-3054:
------------------------------------------
Thanks Vinod, that would have been a circular dependency :).
> Unable to kill submitted jobs
> -----------------------------
>
> Key: MAPREDUCE-3054
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3054
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch
>
>
> Found by Philip Su
> The "mapred job -kill" command
> appears to succeed, but listing the jobs again shows that the job supposedly killed is still there.
> {code}
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> mapred job -kill job_1316203984216_0002
> Killed job job_1316203984216_0002
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3054) Unable to kill submitted jobs
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13113923#comment-13113923 ]
Hadoop QA commented on MAPREDUCE-3054:
--------------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12496356/MAPREDUCE-3054.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 9 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/847//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/847//console
This message is automatically generated.
> Unable to kill submitted jobs
> -----------------------------
>
> Key: MAPREDUCE-3054
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3054
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch
>
>
> Found by Philip Su
> The "mapred job -kill" command
> appears to succeed, but listing the jobs again shows that the job supposedly killed is still there.
> {code}
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> mapred job -kill job_1316203984216_0002
> Killed job job_1316203984216_0002
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-3054) Unable to kill submitted jobs
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar reassigned MAPREDUCE-3054:
----------------------------------------
Assignee: Mahadev konar
> Unable to kill submitted jobs
> -----------------------------
>
> Key: MAPREDUCE-3054
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3054
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Mahadev konar
> Priority: Blocker
>
> Found by Philip Su
> The "mapred job -kill" command
> appears to succeed, but listing the jobs again shows that the job supposedly killed is still there.
> {code}
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> mapred job -kill job_1316203984216_0002
> Killed job job_1316203984216_0002
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3054) Unable to kill submitted jobs
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated MAPREDUCE-3054:
-------------------------------------
Attachment: MAPREDUCE-3054.patch
Preliminary patch. Fixes the issue.
Still need to:
- Add test cases for kill and various states of RMApp
- Fix the RMAppImpl to send a kill signal to RMAppAttemptImpl
> Unable to kill submitted jobs
> -----------------------------
>
> Key: MAPREDUCE-3054
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3054
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3054.patch
>
>
> Found by Philip Su
> The "mapred job -kill" command
> appears to succeed, but listing the jobs again shows that the job supposedly killed is still there.
> {code}
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> mapred job -kill job_1316203984216_0002
> Killed job job_1316203984216_0002
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3054) Unable to kill submitted jobs
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated MAPREDUCE-3054:
-------------------------------------
Attachment: MAPREDUCE-3054.patch
Update patch after merge and adressing comments from Arun.
Addressed 1 and 2.
For 3, currently we use currentAppAttempt for getting diagnostics info from the AM for setting status on UI, so we cant set it to null.
For the tranistion in RMApp, actually its valid. It is possible that there can be a race between an App when submitted and killed such that you can get a kill when its just new.
Also fixed the test failure introduced in MAPREDUCE-3053.
> Unable to kill submitted jobs
> -----------------------------
>
> Key: MAPREDUCE-3054
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3054
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch
>
>
> Found by Philip Su
> The "mapred job -kill" command
> appears to succeed, but listing the jobs again shows that the job supposedly killed is still there.
> {code}
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> mapred job -kill job_1316203984216_0002
> Killed job job_1316203984216_0002
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3054) Unable to kill submitted jobs
Posted by "Vinod Kumar Vavilapalli (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3054:
-----------------------------------------------
Status: Open (was: Patch Available)
Looked at the patch. Some questions and comments on the approach itself.
- With the patch, YarnRunner sends a kill to AM and also a killJob signal to RM in *all* cases. So in all the cases, AM may not be able to gracefully kill itself and write the history file. Absence of history file for all the killed jobs is a broken user experience IMHO. I think we should force-kill on the RM side only when needed.
-- The implementation could be something like this: Check the Application state on RM, if it is running, send a kill-signal to the AM otherwise force-terminate on the RM. Sure, there are corner cases where the client obtains a non-running state, but by the time it sends a force-terminate signal, the app may become running, but losing the (little) information in those rare cases should be bearable.
- Also jobs which are in the queue and get killed have no record in the system( ! ) (except for the fleeting cache in RM). I can't think of any solution other than MAPREDUCE-3061.
- I am tending to rename _ClientRMProtocol.finishApplication()_ to _ClientRMProtocol.forceTerminateApplication()_ to better reflect its semantics. Thoughts?
> Unable to kill submitted jobs
> -----------------------------
>
> Key: MAPREDUCE-3054
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3054
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch
>
>
> Found by Philip Su
> The "mapred job -kill" command
> appears to succeed, but listing the jobs again shows that the job supposedly killed is still there.
> {code}
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> mapred job -kill job_1316203984216_0002
> Killed job job_1316203984216_0002
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3054) Unable to kill submitted jobs
Posted by "Mahadev konar (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated MAPREDUCE-3054:
-------------------------------------
Status: Open (was: Patch Available)
Fixing javadoc. For the test failure, it looks like test-patch is running mvn tests without applying the patch. Will open a jira for that.
> Unable to kill submitted jobs
> -----------------------------
>
> Key: MAPREDUCE-3054
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3054
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch
>
>
> Found by Philip Su
> The "mapred job -kill" command
> appears to succeed, but listing the jobs again shows that the job supposedly killed is still there.
> {code}
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> mapred job -kill job_1316203984216_0002
> Killed job job_1316203984216_0002
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3054) Unable to kill submitted jobs
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13115892#comment-13115892 ]
Hudson commented on MAPREDUCE-3054:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk-Commit #1043 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1043/])
MAPREDUCE-3054. Unable to kill submitted jobs. (mahadev)
mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176600
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientCache.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ResourceMgrDelegate.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientRedirect.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestYARNRunner.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ClientRMProtocol.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/FinishApplicationRequest.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/FinishApplicationResponse.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/KillApplicationRequest.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/KillApplicationResponse.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/FinishApplicationRequestPBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/FinishApplicationResponsePBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/KillApplicationRequestPBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/KillApplicationResponsePBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/client_RM_protocol.proto
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_service_protos.proto
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/client/ClientRMProtocolPBClientImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/service/ClientRMProtocolPBServiceImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/resourcetracker/InlineDispatcher.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/TestRMAppTransitions.java
> Unable to kill submitted jobs
> -----------------------------
>
> Key: MAPREDUCE-3054
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3054
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch
>
>
> Found by Philip Su
> The "mapred job -kill" command
> appears to succeed, but listing the jobs again shows that the job supposedly killed is still there.
> {code}
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> mapred job -kill job_1316203984216_0002
> Killed job job_1316203984216_0002
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3054) Unable to kill submitted jobs
Posted by "Vinod Kumar Vavilapalli (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13114749#comment-13114749 ]
Vinod Kumar Vavilapalli commented on MAPREDUCE-3054:
----------------------------------------------------
That's a little different. Cleaning up of containers(or AM)' process tree is independent and can be triggered once it is clear for the components in question that they are done with their work - via TaskUmbilicalProtocol.done() for the task and AMRMProtocol.unregister() for the AM case.
For the killed jobs, once AM gets the kill-signal, it can gracefully shut down by writing history-file, sending notification url etc, then it can proceed to deregistering with the RM. Once the deregistration comes through, the RM can then direct the NM to make sure the AM is dead.
> Unable to kill submitted jobs
> -----------------------------
>
> Key: MAPREDUCE-3054
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3054
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch
>
>
> Found by Philip Su
> The "mapred job -kill" command
> appears to succeed, but listing the jobs again shows that the job supposedly killed is still there.
> {code}
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> mapred job -kill job_1316203984216_0002
> Killed job job_1316203984216_0002
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3054) Unable to kill submitted jobs
Posted by "Vinod Kumar Vavilapalli (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13116136#comment-13116136 ]
Vinod Kumar Vavilapalli commented on MAPREDUCE-3054:
----------------------------------------------------
I meant via MAPREDUCE-3110.
> Unable to kill submitted jobs
> -----------------------------
>
> Key: MAPREDUCE-3054
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3054
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch
>
>
> Found by Philip Su
> The "mapred job -kill" command
> appears to succeed, but listing the jobs again shows that the job supposedly killed is still there.
> {code}
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> mapred job -kill job_1316203984216_0002
> Killed job job_1316203984216_0002
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3054) Unable to kill submitted jobs
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13115912#comment-13115912 ]
Hudson commented on MAPREDUCE-3054:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #986 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/986/])
MAPREDUCE-3054. Unable to kill submitted jobs. (mahadev)
mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176600
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientCache.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ResourceMgrDelegate.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientRedirect.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestYARNRunner.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ClientRMProtocol.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/FinishApplicationRequest.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/FinishApplicationResponse.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/KillApplicationRequest.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/KillApplicationResponse.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/FinishApplicationRequestPBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/FinishApplicationResponsePBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/KillApplicationRequestPBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/KillApplicationResponsePBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/client_RM_protocol.proto
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_service_protos.proto
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/client/ClientRMProtocolPBClientImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/service/ClientRMProtocolPBServiceImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/resourcetracker/InlineDispatcher.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/TestRMAppTransitions.java
> Unable to kill submitted jobs
> -----------------------------
>
> Key: MAPREDUCE-3054
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3054
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch
>
>
> Found by Philip Su
> The "mapred job -kill" command
> appears to succeed, but listing the jobs again shows that the job supposedly killed is still there.
> {code}
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> mapred job -kill job_1316203984216_0002
> Killed job job_1316203984216_0002
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3054) Unable to kill submitted jobs
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13116399#comment-13116399 ]
Hudson commented on MAPREDUCE-3054:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk #814 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/814/])
MAPREDUCE-3054. Unable to kill submitted jobs. (mahadev)
mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176600
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientCache.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ResourceMgrDelegate.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientRedirect.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestYARNRunner.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ClientRMProtocol.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/FinishApplicationRequest.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/FinishApplicationResponse.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/KillApplicationRequest.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/KillApplicationResponse.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/FinishApplicationRequestPBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/FinishApplicationResponsePBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/KillApplicationRequestPBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/KillApplicationResponsePBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/client_RM_protocol.proto
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_service_protos.proto
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/client/ClientRMProtocolPBClientImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/service/ClientRMProtocolPBServiceImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/resourcetracker/InlineDispatcher.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/TestRMAppTransitions.java
> Unable to kill submitted jobs
> -----------------------------
>
> Key: MAPREDUCE-3054
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3054
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch
>
>
> Found by Philip Su
> The "mapred job -kill" command
> appears to succeed, but listing the jobs again shows that the job supposedly killed is still there.
> {code}
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> mapred job -kill job_1316203984216_0002
> Killed job job_1316203984216_0002
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3054) Unable to kill submitted jobs
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated MAPREDUCE-3054:
-------------------------------------
Status: Patch Available (was: Open)
> Unable to kill submitted jobs
> -----------------------------
>
> Key: MAPREDUCE-3054
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3054
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch
>
>
> Found by Philip Su
> The "mapred job -kill" command
> appears to succeed, but listing the jobs again shows that the job supposedly killed is still there.
> {code}
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> mapred job -kill job_1316203984216_0002
> Killed job job_1316203984216_0002
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3054) Unable to kill submitted jobs
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13116481#comment-13116481 ]
Hudson commented on MAPREDUCE-3054:
-----------------------------------
Integrated in Hadoop-Hdfs-0.23-Build #23 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/23/])
MAPREDUCE-3054. Unable to kill submitted jobs. (mahadev) - Merging r1176600 from trunk
mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176604
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientCache.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ResourceMgrDelegate.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientRedirect.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestYARNRunner.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ClientRMProtocol.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/FinishApplicationRequest.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/FinishApplicationResponse.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/KillApplicationRequest.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/KillApplicationResponse.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/FinishApplicationRequestPBImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/FinishApplicationResponsePBImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/KillApplicationRequestPBImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/KillApplicationResponsePBImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/client_RM_protocol.proto
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_service_protos.proto
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/client/ClientRMProtocolPBClientImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/service/ClientRMProtocolPBServiceImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/TestRPC.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/resourcetracker/InlineDispatcher.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/TestRMAppTransitions.java
> Unable to kill submitted jobs
> -----------------------------
>
> Key: MAPREDUCE-3054
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3054
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch
>
>
> Found by Philip Su
> The "mapred job -kill" command
> appears to succeed, but listing the jobs again shows that the job supposedly killed is still there.
> {code}
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> mapred job -kill job_1316203984216_0002
> Killed job job_1316203984216_0002
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3054) Unable to kill submitted jobs
Posted by "Mahadev konar (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated MAPREDUCE-3054:
-------------------------------------
Status: Patch Available (was: Open)
> Unable to kill submitted jobs
> -----------------------------
>
> Key: MAPREDUCE-3054
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3054
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch
>
>
> Found by Philip Su
> The "mapred job -kill" command
> appears to succeed, but listing the jobs again shows that the job supposedly killed is still there.
> {code}
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> mapred job -kill job_1316203984216_0002
> Killed job job_1316203984216_0002
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3054) Unable to kill submitted jobs
Posted by "Mahadev konar (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated MAPREDUCE-3054:
-------------------------------------
Attachment: MAPREDUCE-3054.patch
@vinod, thanks for the review. I have updated the patch to just use InlineDispatcher. Made changes to InlineDispatcher to suit what I wanted. Also fixed the javadoc warning.
> Unable to kill submitted jobs
> -----------------------------
>
> Key: MAPREDUCE-3054
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3054
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch
>
>
> Found by Philip Su
> The "mapred job -kill" command
> appears to succeed, but listing the jobs again shows that the job supposedly killed is still there.
> {code}
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> mapred job -kill job_1316203984216_0002
> Killed job job_1316203984216_0002
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3054) Unable to kill submitted jobs
Posted by "Vinod Kumar Vavilapalli (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13115309#comment-13115309 ]
Vinod Kumar Vavilapalli commented on MAPREDUCE-3054:
----------------------------------------------------
The sleep for 10 mins is also brittle, especially for medium to big-sized jobs given we already sleep for 5 secs in MRAM before shutdown.
But I wouldn't hold that against this patch. This one clearly solves killing of apps in submitted state. Will open a separate ticket for Job.forceKill() api.
Looked at the patch myself too. Mostly lgtm.
One minor comment: Is BareBonesDispatcher same as InlineDispatcher? If so, you can promote the later to hadoop-yarn-common and drop the earlier.
> Unable to kill submitted jobs
> -----------------------------
>
> Key: MAPREDUCE-3054
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3054
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch
>
>
> Found by Philip Su
> The "mapred job -kill" command
> appears to succeed, but listing the jobs again shows that the job supposedly killed is still there.
> {code}
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> mapred job -kill job_1316203984216_0002
> Killed job job_1316203984216_0002
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3054) Unable to kill submitted jobs
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13115368#comment-13115368 ]
Hadoop QA commented on MAPREDUCE-3054:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12496652/MAPREDUCE-3054.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 18 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.yarn.TestRPC
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/868//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/868//console
This message is automatically generated.
> Unable to kill submitted jobs
> -----------------------------
>
> Key: MAPREDUCE-3054
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3054
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch
>
>
> Found by Philip Su
> The "mapred job -kill" command
> appears to succeed, but listing the jobs again shows that the job supposedly killed is still there.
> {code}
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> mapred job -kill job_1316203984216_0002
> Killed job job_1316203984216_0002
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3054) Unable to kill submitted jobs
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13114350#comment-13114350 ]
Mahadev konar commented on MAPREDUCE-3054:
------------------------------------------
Looks like hudson is running tests on unpatched versions. This patch actually fixes the test case!
> Unable to kill submitted jobs
> -----------------------------
>
> Key: MAPREDUCE-3054
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3054
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch
>
>
> Found by Philip Su
> The "mapred job -kill" command
> appears to succeed, but listing the jobs again shows that the job supposedly killed is still there.
> {code}
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> mapred job -kill job_1316203984216_0002
> Killed job job_1316203984216_0002
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3054) Unable to kill submitted jobs
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13114348#comment-13114348 ]
Hadoop QA commented on MAPREDUCE-3054:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12496407/MAPREDUCE-3054.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 12 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.yarn.TestRPC
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/853//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/853//console
This message is automatically generated.
> Unable to kill submitted jobs
> -----------------------------
>
> Key: MAPREDUCE-3054
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3054
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch
>
>
> Found by Philip Su
> The "mapred job -kill" command
> appears to succeed, but listing the jobs again shows that the job supposedly killed is still there.
> {code}
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> mapred job -kill job_1316203984216_0002
> Killed job job_1316203984216_0002
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3054) Unable to kill submitted jobs
Posted by "Mahadev konar (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated MAPREDUCE-3054:
-------------------------------------
Status: Patch Available (was: Open)
> Unable to kill submitted jobs
> -----------------------------
>
> Key: MAPREDUCE-3054
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3054
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch
>
>
> Found by Philip Su
> The "mapred job -kill" command
> appears to succeed, but listing the jobs again shows that the job supposedly killed is still there.
> {code}
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> mapred job -kill job_1316203984216_0002
> Killed job job_1316203984216_0002
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3054) Unable to kill submitted jobs
Posted by "Vinod Kumar Vavilapalli (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13114754#comment-13114754 ]
Vinod Kumar Vavilapalli commented on MAPREDUCE-3054:
----------------------------------------------------
bq. Also jobs which are in the queue and get killed have no record in the system( ! ) (except for the fleeting cache in RM). I can't think of any solution other than MAPREDUCE-3061.
One idea just stuck me - we can make the JobClient write out the history-file for these jobs which are killed when idling in the queue.
> Unable to kill submitted jobs
> -----------------------------
>
> Key: MAPREDUCE-3054
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3054
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch
>
>
> Found by Philip Su
> The "mapred job -kill" command
> appears to succeed, but listing the jobs again shows that the job supposedly killed is still there.
> {code}
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> mapred job -kill job_1316203984216_0002
> Killed job job_1316203984216_0002
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3054) Unable to kill submitted jobs
Posted by "Vinod Kumar Vavilapalli (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13114801#comment-13114801 ]
Vinod Kumar Vavilapalli commented on MAPREDUCE-3054:
----------------------------------------------------
bq. I dont think we can get away from not sending kill to the RM at all times. I dont think we can rely on AM to be responsive and do the right work. Thoughts?
I agree with the general sentiment here. But the way I'd handle this is by having different UI for graceful-kill and force-terminate: "bin/hadoop job -kill" will do the usual graceful exit. If someone intends to do a "bin/hadoop job -kill -force", they know the risks (of not getting history file, notification etc).
In a way, we are arguing SIGTERM vs SIGKILL semantics here. Sure someone may trap SIGTERM, but SIGTERM only promises a graceful exit if the process is willing to. Otherwise, you go use SIGKILL. We don't need the framework to wrap around SIGTERM and SIGKILL.
> Unable to kill submitted jobs
> -----------------------------
>
> Key: MAPREDUCE-3054
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3054
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch
>
>
> Found by Philip Su
> The "mapred job -kill" command
> appears to succeed, but listing the jobs again shows that the job supposedly killed is still there.
> {code}
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> mapred job -kill job_1316203984216_0002
> Killed job job_1316203984216_0002
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3054) Unable to kill submitted jobs
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-3054:
-------------------------------------
Status: Open (was: Patch Available)
Mahadev, looks good overall - some minor nits:
# YARNRunner should ignore exception from ClientServiceDelegate.kill and send the 'kill' to RM anyway.
# We can remove the 'TODO' which talks abt non-existent handling of KILL in RMAppImpl
# In KillAppAndAttemptTransition, we could also make the currentAppAttempt null i.e. reset it to be safe. But we don't seem to do that anywhere else currently (app failure/completion etc.)
Unrelated - we seem to have a redundant transition for RMApp from NEW -> FAILED. We can only, ever, have SUBMITTED -> FAILED. This is because the scheduler is informed only in SUBMITTED state and then the app can be rejected. Can you pls verify and remove that transition?
> Unable to kill submitted jobs
> -----------------------------
>
> Key: MAPREDUCE-3054
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3054
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch
>
>
> Found by Philip Su
> The "mapred job -kill" command
> appears to succeed, but listing the jobs again shows that the job supposedly killed is still there.
> {code}
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> mapred job -kill job_1316203984216_0002
> Killed job job_1316203984216_0002
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3054) Unable to kill submitted jobs
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated MAPREDUCE-3054:
-------------------------------------
Priority: Blocker (was: Major)
> Unable to kill submitted jobs
> -----------------------------
>
> Key: MAPREDUCE-3054
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3054
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Priority: Blocker
>
> Found by Philip Su
> The "mapred job -kill" command
> appears to succeed, but listing the jobs again shows that the job supposedly killed is still there.
> {code}
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> mapred job -kill job_1316203984216_0002
> Killed job job_1316203984216_0002
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3054) Unable to kill submitted jobs
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13116459#comment-13116459 ]
Hudson commented on MAPREDUCE-3054:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk #844 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/844/])
MAPREDUCE-3054. Unable to kill submitted jobs. (mahadev)
mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176600
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientCache.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ResourceMgrDelegate.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientRedirect.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestYARNRunner.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ClientRMProtocol.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/FinishApplicationRequest.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/FinishApplicationResponse.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/KillApplicationRequest.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/KillApplicationResponse.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/FinishApplicationRequestPBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/FinishApplicationResponsePBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/KillApplicationRequestPBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/KillApplicationResponsePBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/client_RM_protocol.proto
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_service_protos.proto
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/client/ClientRMProtocolPBClientImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/service/ClientRMProtocolPBServiceImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/resourcetracker/InlineDispatcher.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/TestRMAppTransitions.java
> Unable to kill submitted jobs
> -----------------------------
>
> Key: MAPREDUCE-3054
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3054
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch
>
>
> Found by Philip Su
> The "mapred job -kill" command
> appears to succeed, but listing the jobs again shows that the job supposedly killed is still there.
> {code}
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> mapred job -kill job_1316203984216_0002
> Killed job job_1316203984216_0002
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3054) Unable to kill submitted jobs
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13114720#comment-13114720 ]
Mahadev konar commented on MAPREDUCE-3054:
------------------------------------------
I think most of the client commands should be fire and forget. Longer term I think the NodeManager should be sending a SIGTERM, and wait for a while before sending SIGKILL, to let AM/others do some work before getting a SIGKILL.
We always have to force terminate, I am not sure how you can get away from that. We should always be sending a KILL SIGNAL to RM, else we cannot confirm an AM is KILLED. How do you intend to enforce that an AM is dead?
> Unable to kill submitted jobs
> -----------------------------
>
> Key: MAPREDUCE-3054
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3054
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch
>
>
> Found by Philip Su
> The "mapred job -kill" command
> appears to succeed, but listing the jobs again shows that the job supposedly killed is still there.
> {code}
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> mapred job -kill job_1316203984216_0002
> Killed job job_1316203984216_0002
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3054) Unable to kill submitted jobs
Posted by "Mahadev konar (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated MAPREDUCE-3054:
-------------------------------------
Attachment: MAPREDUCE-3054.patch
Ok, here's a compromise. We cant change the job api's to add one to force kill the job for 0.23. So in my patch, the kill command will send a kill to RM if the job is not running, if RUNNING, it will send a kill to AM and wait for 10 seconds for it to get killed. If it doesnt get killed, it will send a force kill to RM.
Also, made a couple of other changes:
- added a little more testing
- changed the finishApplication calls to forceKillApplication in RM
- also changed the response/request names to be clear on what they are doing.
Seems reasonable?
> Unable to kill submitted jobs
> -----------------------------
>
> Key: MAPREDUCE-3054
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3054
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch
>
>
> Found by Philip Su
> The "mapred job -kill" command
> appears to succeed, but listing the jobs again shows that the job supposedly killed is still there.
> {code}
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> mapred job -kill job_1316203984216_0002
> Killed job job_1316203984216_0002
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3054) Unable to kill submitted jobs
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13115013#comment-13115013 ]
Hadoop QA commented on MAPREDUCE-3054:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12496581/MAPREDUCE-3054.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 15 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated 2 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.yarn.TestRPC
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/862//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/862//console
This message is automatically generated.
> Unable to kill submitted jobs
> -----------------------------
>
> Key: MAPREDUCE-3054
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3054
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch
>
>
> Found by Philip Su
> The "mapred job -kill" command
> appears to succeed, but listing the jobs again shows that the job supposedly killed is still there.
> {code}
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> mapred job -kill job_1316203984216_0002
> Killed job job_1316203984216_0002
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3054) Unable to kill submitted jobs
Posted by "Arun C Murthy (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13115035#comment-13115035 ]
Arun C Murthy commented on MAPREDUCE-3054:
------------------------------------------
+1
> Unable to kill submitted jobs
> -----------------------------
>
> Key: MAPREDUCE-3054
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3054
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch
>
>
> Found by Philip Su
> The "mapred job -kill" command
> appears to succeed, but listing the jobs again shows that the job supposedly killed is still there.
> {code}
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> mapred job -kill job_1316203984216_0002
> Killed job job_1316203984216_0002
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3054) Unable to kill submitted jobs
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13114759#comment-13114759 ]
Mahadev konar commented on MAPREDUCE-3054:
------------------------------------------
@Vinod, Its probably not different but that argument is for another day :).
For the AM getting KILL signal, we can never be sure that the AM will call finishapplicationMaster. How would you enforce that? I dont think we can get away from not sending kill to the RM at *all* times. I dont think we can rely on AM to be responsive and do the right work. Thoughts?
On the renaming of the finishAPplication, I do agree we should change that.
> Unable to kill submitted jobs
> -----------------------------
>
> Key: MAPREDUCE-3054
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3054
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch
>
>
> Found by Philip Su
> The "mapred job -kill" command
> appears to succeed, but listing the jobs again shows that the job supposedly killed is still there.
> {code}
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> mapred job -kill job_1316203984216_0002
> Killed job job_1316203984216_0002
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3054) Unable to kill submitted jobs
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated MAPREDUCE-3054:
-------------------------------------
Fix Version/s: 0.23.0
> Unable to kill submitted jobs
> -----------------------------
>
> Key: MAPREDUCE-3054
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3054
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
>
> Found by Philip Su
> The "mapred job -kill" command
> appears to succeed, but listing the jobs again shows that the job supposedly killed is still there.
> {code}
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> mapred job -kill job_1316203984216_0002
> Killed job job_1316203984216_0002
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3054) Unable to kill submitted jobs
Posted by "Vinod Kumar Vavilapalli (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13116130#comment-13116130 ]
Vinod Kumar Vavilapalli commented on MAPREDUCE-3054:
----------------------------------------------------
bq. -1 core tests. The patch failed these unit tests:
org.apache.hadoop.yarn.TestRPC
The patch broke TestRPC on Jenkins. The exception trace seems to be dependant on the jvm used. I'll fix it via MAPREDUCE-3053.
> Unable to kill submitted jobs
> -----------------------------
>
> Key: MAPREDUCE-3054
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3054
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch
>
>
> Found by Philip Su
> The "mapred job -kill" command
> appears to succeed, but listing the jobs again shows that the job supposedly killed is still there.
> {code}
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> mapred job -kill job_1316203984216_0002
> Killed job job_1316203984216_0002
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3054) Unable to kill submitted jobs
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13115891#comment-13115891 ]
Hudson commented on MAPREDUCE-3054:
-----------------------------------
Integrated in Hadoop-Common-trunk-Commit #965 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/965/])
MAPREDUCE-3054. Unable to kill submitted jobs. (mahadev)
mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176600
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientCache.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ResourceMgrDelegate.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientRedirect.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestYARNRunner.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ClientRMProtocol.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/FinishApplicationRequest.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/FinishApplicationResponse.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/KillApplicationRequest.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/KillApplicationResponse.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/FinishApplicationRequestPBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/FinishApplicationResponsePBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/KillApplicationRequestPBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/KillApplicationResponsePBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/client_RM_protocol.proto
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_service_protos.proto
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/client/ClientRMProtocolPBClientImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/service/ClientRMProtocolPBServiceImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/resourcetracker/InlineDispatcher.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/TestRMAppTransitions.java
> Unable to kill submitted jobs
> -----------------------------
>
> Key: MAPREDUCE-3054
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3054
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch
>
>
> Found by Philip Su
> The "mapred job -kill" command
> appears to succeed, but listing the jobs again shows that the job supposedly killed is still there.
> {code}
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> mapred job -kill job_1316203984216_0002
> Killed job job_1316203984216_0002
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3054) Unable to kill submitted jobs
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13115046#comment-13115046 ]
Hadoop QA commented on MAPREDUCE-3054:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12496588/MAPREDUCE-3054.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 15 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated 1 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.yarn.TestRPC
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/863//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/863//console
This message is automatically generated.
> Unable to kill submitted jobs
> -----------------------------
>
> Key: MAPREDUCE-3054
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3054
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch
>
>
> Found by Philip Su
> The "mapred job -kill" command
> appears to succeed, but listing the jobs again shows that the job supposedly killed is still there.
> {code}
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> mapred job -kill job_1316203984216_0002
> Killed job job_1316203984216_0002
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3054) Unable to kill submitted jobs
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated MAPREDUCE-3054:
-------------------------------------
Status: Patch Available (was: Open)
> Unable to kill submitted jobs
> -----------------------------
>
> Key: MAPREDUCE-3054
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3054
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch
>
>
> Found by Philip Su
> The "mapred job -kill" command
> appears to succeed, but listing the jobs again shows that the job supposedly killed is still there.
> {code}
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> mapred job -kill job_1316203984216_0002
> Killed job job_1316203984216_0002
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3054) Unable to kill submitted jobs
Posted by "Mahadev konar (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated MAPREDUCE-3054:
-------------------------------------
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Just committed this. Thanks Arun and Vinod for a detailed review.
> Unable to kill submitted jobs
> -----------------------------
>
> Key: MAPREDUCE-3054
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3054
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch, MAPREDUCE-3054.patch
>
>
> Found by Philip Su
> The "mapred job -kill" command
> appears to succeed, but listing the jobs again shows that the job supposedly killed is still there.
> {code}
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> mapred job -kill job_1316203984216_0002
> Killed job job_1316203984216_0002
> mapred job -list
> Total jobs:2
> JobId State StartTime UserName Queue Priority SchedulingInfo
> job_1316203984216_0002 PREP 1316204924937 hadoopqa default NORMAL
> job_1316203984216_0001 PREP 1316204031206 hadoopqa default NORMAL
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira