You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Vinod Kumar Vavilapalli (Created) (JIRA)" <ji...@apache.org> on 2011/10/25 07:40:32 UTC
[jira] [Created] (MAPREDUCE-3256) Authorization checks needed for
AM->NM and AM->RM protocols
Authorization checks needed for AM->NM and AM->RM protocols
-----------------------------------------------------------
Key: MAPREDUCE-3256
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3256
Project: Hadoop Map/Reduce
Issue Type: Sub-task
Reporter: Vinod Kumar Vavilapalli
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3256) Authorization checks needed for
AM->NM protocol
Posted by "Vinod Kumar Vavilapalli (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3256:
-----------------------------------------------
Attachment: MAPREDUCE-3256-20111029.1.txt
> Authorization checks needed for AM->NM protocol
> -----------------------------------------------
>
> Key: MAPREDUCE-3256
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3256
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: applicationmaster, mrv2, nodemanager, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3256-20111028.1.txt, MAPREDUCE-3256-20111028.2.txt, MAPREDUCE-3256-20111028.2_same, MAPREDUCE-3256-20111029.1.txt, MAPREDUCE-3256-20111029.txt
>
>
> We already authenticate requests to NM from any AM. We also need to authorize the requests, otherwise a rogue AM, *but with proper tokens and thus authenticated to talk to NM*, could either launch or kill a container with different ContainerID. We have two options:
> - Remove the explicit passing of the ContainerId as part of the API and instead get it from the RPC layer. In this case, we will need a ContainerToken for each container.
> - Do explicit authorization checks without relying on getting ContainerID from the RPC.
> One ContainerToken per container is a serious restriction. We anyways want to be able to use application-ACLS to, say, stop containers owned by others. So I am going to take the later route of explicit checks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3256) Authorization checks needed for
AM->NM protocol
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13138820#comment-13138820 ]
Hadoop QA commented on MAPREDUCE-3256:
--------------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12501363/MAPREDUCE-3256-20111028.2_same
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 10 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1200//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1200//console
This message is automatically generated.
> Authorization checks needed for AM->NM protocol
> -----------------------------------------------
>
> Key: MAPREDUCE-3256
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3256
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: applicationmaster, mrv2, nodemanager, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3256-20111028.1.txt, MAPREDUCE-3256-20111028.2.txt, MAPREDUCE-3256-20111028.2_same
>
>
> We already authenticate requests to NM from any AM. We also need to authorize the requests, otherwise a rogue AM, *but with proper tokens and thus authenticated to talk to NM*, could either launch or kill a container with different ContainerID. We have two options:
> - Remove the explicit passing of the ContainerId as part of the API and instead get it from the RPC layer. In this case, we will need a ContainerToken for each container.
> - Do explicit authorization checks without relying on getting ContainerID from the RPC.
> One ContainerToken per container is a serious restriction. We anyways want to be able to use application-ACLS to, say, stop containers owned by others. So I am going to take the later route of explicit checks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3256) Authorization checks needed for
AM->NM protocol
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139175#comment-13139175 ]
Hudson commented on MAPREDUCE-3256:
-----------------------------------
Integrated in Hadoop-Mapreduce-0.23-Commit #109 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/109/])
Merge -c 1194850 from trunk to branch-0.23 to fix MAPREDUCE-3256.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1194851
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/launcher/ContainerLauncherImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestContainerLauncher.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestFail.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/Resource.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ResourcePBImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/ContainerTokenIdentifier.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/ContainerTokenSelector.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/BuilderUtils.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/security/ContainerTokenSecretManager.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/ContainerManagerImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/security/LocalizerTokenSelector.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/amlauncher/AMLauncher.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationMasterLauncher.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/TestContainerManagerSecurity.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/TestContainerTokenSecretManager.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/resources/log4j.properties
> Authorization checks needed for AM->NM protocol
> -----------------------------------------------
>
> Key: MAPREDUCE-3256
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3256
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: applicationmaster, mrv2, nodemanager, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3256-20111028.1.txt, MAPREDUCE-3256-20111028.2.txt, MAPREDUCE-3256-20111028.2_same, MAPREDUCE-3256-20111029.1.txt, MAPREDUCE-3256-20111029.txt
>
>
> We already authenticate requests to NM from any AM. We also need to authorize the requests, otherwise a rogue AM, *but with proper tokens and thus authenticated to talk to NM*, could either launch or kill a container with different ContainerID. We have two options:
> - Remove the explicit passing of the ContainerId as part of the API and instead get it from the RPC layer. In this case, we will need a ContainerToken for each container.
> - Do explicit authorization checks without relying on getting ContainerID from the RPC.
> One ContainerToken per container is a serious restriction. We anyways want to be able to use application-ACLS to, say, stop containers owned by others. So I am going to take the later route of explicit checks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3256) Authorization checks needed for
AM->NM protocol
Posted by "Arun C Murthy (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-3256:
-------------------------------------
Attachment: MAPREDUCE-3256-20111028.2_same
Same patch.
> Authorization checks needed for AM->NM protocol
> -----------------------------------------------
>
> Key: MAPREDUCE-3256
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3256
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: applicationmaster, mrv2, nodemanager, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3256-20111028.1.txt, MAPREDUCE-3256-20111028.2.txt, MAPREDUCE-3256-20111028.2_same
>
>
> We already authenticate requests to NM from any AM. We also need to authorize the requests, otherwise a rogue AM, *but with proper tokens and thus authenticated to talk to NM*, could either launch or kill a container with different ContainerID. We have two options:
> - Remove the explicit passing of the ContainerId as part of the API and instead get it from the RPC layer. In this case, we will need a ContainerToken for each container.
> - Do explicit authorization checks without relying on getting ContainerID from the RPC.
> One ContainerToken per container is a serious restriction. We anyways want to be able to use application-ACLS to, say, stop containers owned by others. So I am going to take the later route of explicit checks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3256) Authorization checks needed for
AM->NM protocol
Posted by "Vinod Kumar Vavilapalli (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139191#comment-13139191 ]
Vinod Kumar Vavilapalli commented on MAPREDUCE-3256:
----------------------------------------------------
bq. [..] run a series of java-MR/streaming tests on a secure setup.
Thanks a bunch Arun for validating it on cluster!
> Authorization checks needed for AM->NM protocol
> -----------------------------------------------
>
> Key: MAPREDUCE-3256
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3256
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: applicationmaster, mrv2, nodemanager, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3256-20111028.1.txt, MAPREDUCE-3256-20111028.2.txt, MAPREDUCE-3256-20111028.2_same, MAPREDUCE-3256-20111029.1.txt, MAPREDUCE-3256-20111029.txt
>
>
> We already authenticate requests to NM from any AM. We also need to authorize the requests, otherwise a rogue AM, *but with proper tokens and thus authenticated to talk to NM*, could either launch or kill a container with different ContainerID. We have two options:
> - Remove the explicit passing of the ContainerId as part of the API and instead get it from the RPC layer. In this case, we will need a ContainerToken for each container.
> - Do explicit authorization checks without relying on getting ContainerID from the RPC.
> One ContainerToken per container is a serious restriction. We anyways want to be able to use application-ACLS to, say, stop containers owned by others. So I am going to take the later route of explicit checks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3256) Authorization checks needed for
AM->NM protocol
Posted by "Arun C Murthy (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-3256:
-------------------------------------
Status: Open (was: Patch Available)
Vinod, with this patch I'm seeing issues on a secure cluster:
{noformat}
Container launch failed for container_1319838127023_0002_01_000032 : RemoteTrace:
at LocalTrace:
org.apache.hadoop.yarn.exceptions.impl.pb.YarnRemoteExceptionPBImpl: Unauthorized request to start container.
Expected containerId: container_1319838127023_0002_01_000017 Found: container_1319838127023_0002_01_000032
at org.apache.hadoop.yarn.ipc.ProtoOverHadoopRpcEngine$Invoker.invoke(ProtoOverHadoopRpcEngine.java:151)
at $Proxy20.startContainer(Unknown Source)
at org.apache.hadoop.yarn.api.impl.pb.client.ContainerManagerPBClientImpl.startContainer(ContainerManagerPBClientImpl.java:81)
at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$EventProcessor.run(ContainerLauncherImpl.java:278)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
{noformat}
> Authorization checks needed for AM->NM protocol
> -----------------------------------------------
>
> Key: MAPREDUCE-3256
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3256
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: applicationmaster, mrv2, nodemanager, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3256-20111028.1.txt, MAPREDUCE-3256-20111028.2.txt, MAPREDUCE-3256-20111028.2_same
>
>
> We already authenticate requests to NM from any AM. We also need to authorize the requests, otherwise a rogue AM, *but with proper tokens and thus authenticated to talk to NM*, could either launch or kill a container with different ContainerID. We have two options:
> - Remove the explicit passing of the ContainerId as part of the API and instead get it from the RPC layer. In this case, we will need a ContainerToken for each container.
> - Do explicit authorization checks without relying on getting ContainerID from the RPC.
> One ContainerToken per container is a serious restriction. We anyways want to be able to use application-ACLS to, say, stop containers owned by others. So I am going to take the later route of explicit checks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3256) Authorization checks needed for
AM->NM protocol
Posted by "Vinod Kumar Vavilapalli (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3256:
-----------------------------------------------
Status: Open (was: Patch Available)
> Authorization checks needed for AM->NM protocol
> -----------------------------------------------
>
> Key: MAPREDUCE-3256
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3256
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: applicationmaster, mrv2, nodemanager, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3256-20111028.1.txt
>
>
> We already authenticate requests to NM from any AM. We also need to authorize the requests, otherwise a rogue AM, *but with proper tokens and thus authenticated to talk to NM*, could either launch or kill a container with different ContainerID. We have two options:
> - Remove the explicit passing of the ContainerId as part of the API and instead get it from the RPC layer. In this case, we will need a ContainerToken for each container.
> - Do explicit authorization checks without relying on getting ContainerID from the RPC.
> One ContainerToken per container is a serious restriction. We anyways want to be able to use application-ACLS to, say, stop containers owned by others. So I am going to take the later route of explicit checks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3256) Authorization checks needed for
AM->NM protocol
Posted by "Vinod Kumar Vavilapalli (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3256:
-----------------------------------------------
Attachment: MAPREDUCE-3256-20111028.2.txt
Fixing test.
> Authorization checks needed for AM->NM protocol
> -----------------------------------------------
>
> Key: MAPREDUCE-3256
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3256
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: applicationmaster, mrv2, nodemanager, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3256-20111028.1.txt, MAPREDUCE-3256-20111028.2.txt
>
>
> We already authenticate requests to NM from any AM. We also need to authorize the requests, otherwise a rogue AM, *but with proper tokens and thus authenticated to talk to NM*, could either launch or kill a container with different ContainerID. We have two options:
> - Remove the explicit passing of the ContainerId as part of the API and instead get it from the RPC layer. In this case, we will need a ContainerToken for each container.
> - Do explicit authorization checks without relying on getting ContainerID from the RPC.
> One ContainerToken per container is a serious restriction. We anyways want to be able to use application-ACLS to, say, stop containers owned by others. So I am going to take the later route of explicit checks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3256) Authorization checks needed for
AM->NM protocol
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139144#comment-13139144 ]
Hadoop QA commented on MAPREDUCE-3256:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12501435/MAPREDUCE-3256-20111029.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 16 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1203//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1203//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-app.html
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1203//console
This message is automatically generated.
> Authorization checks needed for AM->NM protocol
> -----------------------------------------------
>
> Key: MAPREDUCE-3256
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3256
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: applicationmaster, mrv2, nodemanager, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3256-20111028.1.txt, MAPREDUCE-3256-20111028.2.txt, MAPREDUCE-3256-20111028.2_same, MAPREDUCE-3256-20111029.txt
>
>
> We already authenticate requests to NM from any AM. We also need to authorize the requests, otherwise a rogue AM, *but with proper tokens and thus authenticated to talk to NM*, could either launch or kill a container with different ContainerID. We have two options:
> - Remove the explicit passing of the ContainerId as part of the API and instead get it from the RPC layer. In this case, we will need a ContainerToken for each container.
> - Do explicit authorization checks without relying on getting ContainerID from the RPC.
> One ContainerToken per container is a serious restriction. We anyways want to be able to use application-ACLS to, say, stop containers owned by others. So I am going to take the later route of explicit checks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3256) Authorization checks needed for
AM->NM protocol
Posted by "Vinod Kumar Vavilapalli (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139123#comment-13139123 ]
Vinod Kumar Vavilapalli commented on MAPREDUCE-3256:
----------------------------------------------------
That bug clearly demonstrates that my patch is working :) And of course, that there are more bugs to fix in AM.
AM is using the same tokens for starting/stopping various containers! Which is exactly what we are trying to prevent. Working on it.
> Authorization checks needed for AM->NM protocol
> -----------------------------------------------
>
> Key: MAPREDUCE-3256
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3256
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: applicationmaster, mrv2, nodemanager, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3256-20111028.1.txt, MAPREDUCE-3256-20111028.2.txt, MAPREDUCE-3256-20111028.2_same
>
>
> We already authenticate requests to NM from any AM. We also need to authorize the requests, otherwise a rogue AM, *but with proper tokens and thus authenticated to talk to NM*, could either launch or kill a container with different ContainerID. We have two options:
> - Remove the explicit passing of the ContainerId as part of the API and instead get it from the RPC layer. In this case, we will need a ContainerToken for each container.
> - Do explicit authorization checks without relying on getting ContainerID from the RPC.
> One ContainerToken per container is a serious restriction. We anyways want to be able to use application-ACLS to, say, stop containers owned by others. So I am going to take the later route of explicit checks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3256) Authorization checks needed for
AM->NM protocol
Posted by "Vinod Kumar Vavilapalli (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3256:
-----------------------------------------------
Status: Patch Available (was: Open)
> Authorization checks needed for AM->NM protocol
> -----------------------------------------------
>
> Key: MAPREDUCE-3256
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3256
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: applicationmaster, mrv2, nodemanager, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3256-20111028.1.txt, MAPREDUCE-3256-20111028.2.txt
>
>
> We already authenticate requests to NM from any AM. We also need to authorize the requests, otherwise a rogue AM, *but with proper tokens and thus authenticated to talk to NM*, could either launch or kill a container with different ContainerID. We have two options:
> - Remove the explicit passing of the ContainerId as part of the API and instead get it from the RPC layer. In this case, we will need a ContainerToken for each container.
> - Do explicit authorization checks without relying on getting ContainerID from the RPC.
> One ContainerToken per container is a serious restriction. We anyways want to be able to use application-ACLS to, say, stop containers owned by others. So I am going to take the later route of explicit checks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3256) Authorization checks needed for
AM->NM protocol
Posted by "Vinod Kumar Vavilapalli (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3256:
-----------------------------------------------
Description:
We already authenticate requests to NM from any AM. We also need to authorize the requests, otherwise a rogue AM, *but with proper tokens and thus authenticated to talk to NM*, could either launch or kill a container with different ContainerID. We have two options:
- Remove the explicit passing of the ContainerId as part of the API and instead get it from the RPC layer. In this case, we will need a ContainerToken for each container.
- Do explicit authorization checks without relying on getting ContainerID from the RPC.
One ContainerToken per container is a serious restriction. We anyways want to be able to use application-ACLS to, say, stop containers owned by others. So I am going to take the later route of explicit checks.
was:We already authenticate requests to NM and RM from any AM. We also need to authorize the requests, otherwise a rogue AM could either launch or kill a container with different ContainerID or masquerade as another AM and illegally obtain containers meant for others.
Summary: Authorization checks needed for AM->NM protocol (was: Authorization checks needed for AM->NM and AM->RM protocols)
Okay, I am realizing that each protocol is a good chunk of work in itself. I am going to do each of them separately. Using this ticket for ContainerManager protocol related authorization checks.
> Authorization checks needed for AM->NM protocol
> -----------------------------------------------
>
> Key: MAPREDUCE-3256
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3256
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: applicationmaster, nodemanager, resourcemanager, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Priority: Blocker
> Fix For: 0.23.0
>
>
> We already authenticate requests to NM from any AM. We also need to authorize the requests, otherwise a rogue AM, *but with proper tokens and thus authenticated to talk to NM*, could either launch or kill a container with different ContainerID. We have two options:
> - Remove the explicit passing of the ContainerId as part of the API and instead get it from the RPC layer. In this case, we will need a ContainerToken for each container.
> - Do explicit authorization checks without relying on getting ContainerID from the RPC.
> One ContainerToken per container is a serious restriction. We anyways want to be able to use application-ACLS to, say, stop containers owned by others. So I am going to take the later route of explicit checks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3256) Authorization checks needed for
AM->NM protocol
Posted by "Vinod Kumar Vavilapalli (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3256:
-----------------------------------------------
Component/s: (was: resourcemanager)
mrv2
> Authorization checks needed for AM->NM protocol
> -----------------------------------------------
>
> Key: MAPREDUCE-3256
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3256
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: applicationmaster, mrv2, nodemanager, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Priority: Blocker
> Fix For: 0.23.0
>
>
> We already authenticate requests to NM from any AM. We also need to authorize the requests, otherwise a rogue AM, *but with proper tokens and thus authenticated to talk to NM*, could either launch or kill a container with different ContainerID. We have two options:
> - Remove the explicit passing of the ContainerId as part of the API and instead get it from the RPC layer. In this case, we will need a ContainerToken for each container.
> - Do explicit authorization checks without relying on getting ContainerID from the RPC.
> One ContainerToken per container is a serious restriction. We anyways want to be able to use application-ACLS to, say, stop containers owned by others. So I am going to take the later route of explicit checks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3256) Authorization checks needed for
AM->NM and AM->RM protocols
Posted by "Vinod Kumar Vavilapalli (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3256:
-----------------------------------------------
Component/s: resourcemanager
nodemanager
applicationmaster
Description: We already authenticate requests to NM and RM from any AM. We also need to authorize the requests, otherwise a rogue AM could either launch or kill a container with different ContainerID or masquerade as another AM and illegally obtain containers meant for others.
Priority: Blocker (was: Major)
Affects Version/s: 0.23.0
> Authorization checks needed for AM->NM and AM->RM protocols
> -----------------------------------------------------------
>
> Key: MAPREDUCE-3256
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3256
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: applicationmaster, nodemanager, resourcemanager, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Priority: Blocker
> Fix For: 0.23.0
>
>
> We already authenticate requests to NM and RM from any AM. We also need to authorize the requests, otherwise a rogue AM could either launch or kill a container with different ContainerID or masquerade as another AM and illegally obtain containers meant for others.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3256) Authorization checks needed for
AM->NM protocol
Posted by "Vinod Kumar Vavilapalli (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3256:
-----------------------------------------------
Status: Patch Available (was: Open)
> Authorization checks needed for AM->NM protocol
> -----------------------------------------------
>
> Key: MAPREDUCE-3256
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3256
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: applicationmaster, mrv2, nodemanager, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3256-20111028.1.txt, MAPREDUCE-3256-20111028.2.txt, MAPREDUCE-3256-20111028.2_same, MAPREDUCE-3256-20111029.txt
>
>
> We already authenticate requests to NM from any AM. We also need to authorize the requests, otherwise a rogue AM, *but with proper tokens and thus authenticated to talk to NM*, could either launch or kill a container with different ContainerID. We have two options:
> - Remove the explicit passing of the ContainerId as part of the API and instead get it from the RPC layer. In this case, we will need a ContainerToken for each container.
> - Do explicit authorization checks without relying on getting ContainerID from the RPC.
> One ContainerToken per container is a serious restriction. We anyways want to be able to use application-ACLS to, say, stop containers owned by others. So I am going to take the later route of explicit checks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3256) Authorization checks needed for
AM->NM protocol
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139166#comment-13139166 ]
Hudson commented on MAPREDUCE-3256:
-----------------------------------
Integrated in Hadoop-Common-0.23-Commit #102 (See [https://builds.apache.org/job/Hadoop-Common-0.23-Commit/102/])
Merge -c 1194850 from trunk to branch-0.23 to fix MAPREDUCE-3256.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1194851
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/launcher/ContainerLauncherImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestContainerLauncher.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestFail.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/Resource.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ResourcePBImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/ContainerTokenIdentifier.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/ContainerTokenSelector.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/BuilderUtils.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/security/ContainerTokenSecretManager.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/ContainerManagerImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/security/LocalizerTokenSelector.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/amlauncher/AMLauncher.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationMasterLauncher.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/TestContainerManagerSecurity.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/TestContainerTokenSecretManager.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/resources/log4j.properties
> Authorization checks needed for AM->NM protocol
> -----------------------------------------------
>
> Key: MAPREDUCE-3256
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3256
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: applicationmaster, mrv2, nodemanager, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3256-20111028.1.txt, MAPREDUCE-3256-20111028.2.txt, MAPREDUCE-3256-20111028.2_same, MAPREDUCE-3256-20111029.1.txt, MAPREDUCE-3256-20111029.txt
>
>
> We already authenticate requests to NM from any AM. We also need to authorize the requests, otherwise a rogue AM, *but with proper tokens and thus authenticated to talk to NM*, could either launch or kill a container with different ContainerID. We have two options:
> - Remove the explicit passing of the ContainerId as part of the API and instead get it from the RPC layer. In this case, we will need a ContainerToken for each container.
> - Do explicit authorization checks without relying on getting ContainerID from the RPC.
> One ContainerToken per container is a serious restriction. We anyways want to be able to use application-ACLS to, say, stop containers owned by others. So I am going to take the later route of explicit checks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3256) Authorization checks needed for
AM->NM protocol
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139149#comment-13139149 ]
Hadoop QA commented on MAPREDUCE-3256:
--------------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12501436/MAPREDUCE-3256-20111029.1.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 16 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1204//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1204//console
This message is automatically generated.
> Authorization checks needed for AM->NM protocol
> -----------------------------------------------
>
> Key: MAPREDUCE-3256
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3256
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: applicationmaster, mrv2, nodemanager, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3256-20111028.1.txt, MAPREDUCE-3256-20111028.2.txt, MAPREDUCE-3256-20111028.2_same, MAPREDUCE-3256-20111029.1.txt, MAPREDUCE-3256-20111029.txt
>
>
> We already authenticate requests to NM from any AM. We also need to authorize the requests, otherwise a rogue AM, *but with proper tokens and thus authenticated to talk to NM*, could either launch or kill a container with different ContainerID. We have two options:
> - Remove the explicit passing of the ContainerId as part of the API and instead get it from the RPC layer. In this case, we will need a ContainerToken for each container.
> - Do explicit authorization checks without relying on getting ContainerID from the RPC.
> One ContainerToken per container is a serious restriction. We anyways want to be able to use application-ACLS to, say, stop containers owned by others. So I am going to take the later route of explicit checks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3256) Authorization checks needed for
AM->NM protocol
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139299#comment-13139299 ]
Hudson commented on MAPREDUCE-3256:
-----------------------------------
Integrated in Hadoop-Mapreduce-0.23-Build #72 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/72/])
Merge -c 1194850 from trunk to branch-0.23 to fix MAPREDUCE-3256.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1194851
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/launcher/ContainerLauncherImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestContainerLauncher.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestFail.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/Resource.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ResourcePBImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/ContainerTokenIdentifier.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/ContainerTokenSelector.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/BuilderUtils.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/security/ContainerTokenSecretManager.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/ContainerManagerImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/security/LocalizerTokenSelector.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/amlauncher/AMLauncher.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationMasterLauncher.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/TestContainerManagerSecurity.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/TestContainerTokenSecretManager.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/resources/log4j.properties
> Authorization checks needed for AM->NM protocol
> -----------------------------------------------
>
> Key: MAPREDUCE-3256
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3256
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: applicationmaster, mrv2, nodemanager, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3256-20111028.1.txt, MAPREDUCE-3256-20111028.2.txt, MAPREDUCE-3256-20111028.2_same, MAPREDUCE-3256-20111029.1.txt, MAPREDUCE-3256-20111029.txt
>
>
> We already authenticate requests to NM from any AM. We also need to authorize the requests, otherwise a rogue AM, *but with proper tokens and thus authenticated to talk to NM*, could either launch or kill a container with different ContainerID. We have two options:
> - Remove the explicit passing of the ContainerId as part of the API and instead get it from the RPC layer. In this case, we will need a ContainerToken for each container.
> - Do explicit authorization checks without relying on getting ContainerID from the RPC.
> One ContainerToken per container is a serious restriction. We anyways want to be able to use application-ACLS to, say, stop containers owned by others. So I am going to take the later route of explicit checks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3256) Authorization checks needed for
AM->NM protocol
Posted by "Arun C Murthy (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-3256:
-------------------------------------
Status: Patch Available (was: Open)
> Authorization checks needed for AM->NM protocol
> -----------------------------------------------
>
> Key: MAPREDUCE-3256
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3256
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: applicationmaster, mrv2, nodemanager, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3256-20111028.1.txt, MAPREDUCE-3256-20111028.2.txt, MAPREDUCE-3256-20111028.2_same
>
>
> We already authenticate requests to NM from any AM. We also need to authorize the requests, otherwise a rogue AM, *but with proper tokens and thus authenticated to talk to NM*, could either launch or kill a container with different ContainerID. We have two options:
> - Remove the explicit passing of the ContainerId as part of the API and instead get it from the RPC layer. In this case, we will need a ContainerToken for each container.
> - Do explicit authorization checks without relying on getting ContainerID from the RPC.
> One ContainerToken per container is a serious restriction. We anyways want to be able to use application-ACLS to, say, stop containers owned by others. So I am going to take the later route of explicit checks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3256) Authorization checks needed for
AM->NM protocol
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139171#comment-13139171 ]
Hudson commented on MAPREDUCE-3256:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #1220 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1220/])
MAPREDUCE-3256. Added authorization checks for the protocol between NodeManager and ApplicationMaster. Contributed by Vinod K V.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1194850
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/launcher/ContainerLauncherImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestContainerLauncher.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestFail.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/Resource.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ResourcePBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/ContainerTokenIdentifier.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/ContainerTokenSelector.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/BuilderUtils.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/security/ContainerTokenSecretManager.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/ContainerManagerImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/security/LocalizerTokenSelector.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/amlauncher/AMLauncher.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationMasterLauncher.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/TestContainerManagerSecurity.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/TestContainerTokenSecretManager.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/resources/log4j.properties
> Authorization checks needed for AM->NM protocol
> -----------------------------------------------
>
> Key: MAPREDUCE-3256
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3256
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: applicationmaster, mrv2, nodemanager, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3256-20111028.1.txt, MAPREDUCE-3256-20111028.2.txt, MAPREDUCE-3256-20111028.2_same, MAPREDUCE-3256-20111029.1.txt, MAPREDUCE-3256-20111029.txt
>
>
> We already authenticate requests to NM from any AM. We also need to authorize the requests, otherwise a rogue AM, *but with proper tokens and thus authenticated to talk to NM*, could either launch or kill a container with different ContainerID. We have two options:
> - Remove the explicit passing of the ContainerId as part of the API and instead get it from the RPC layer. In this case, we will need a ContainerToken for each container.
> - Do explicit authorization checks without relying on getting ContainerID from the RPC.
> One ContainerToken per container is a serious restriction. We anyways want to be able to use application-ACLS to, say, stop containers owned by others. So I am going to take the later route of explicit checks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3256) Authorization checks needed for
AM->NM protocol
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139275#comment-13139275 ]
Hudson commented on MAPREDUCE-3256:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk #878 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/878/])
MAPREDUCE-3256. Added authorization checks for the protocol between NodeManager and ApplicationMaster. Contributed by Vinod K V.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1194850
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/launcher/ContainerLauncherImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestContainerLauncher.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestFail.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/Resource.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ResourcePBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/ContainerTokenIdentifier.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/ContainerTokenSelector.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/BuilderUtils.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/security/ContainerTokenSecretManager.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/ContainerManagerImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/security/LocalizerTokenSelector.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/amlauncher/AMLauncher.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationMasterLauncher.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/TestContainerManagerSecurity.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/TestContainerTokenSecretManager.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/resources/log4j.properties
> Authorization checks needed for AM->NM protocol
> -----------------------------------------------
>
> Key: MAPREDUCE-3256
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3256
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: applicationmaster, mrv2, nodemanager, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3256-20111028.1.txt, MAPREDUCE-3256-20111028.2.txt, MAPREDUCE-3256-20111028.2_same, MAPREDUCE-3256-20111029.1.txt, MAPREDUCE-3256-20111029.txt
>
>
> We already authenticate requests to NM from any AM. We also need to authorize the requests, otherwise a rogue AM, *but with proper tokens and thus authenticated to talk to NM*, could either launch or kill a container with different ContainerID. We have two options:
> - Remove the explicit passing of the ContainerId as part of the API and instead get it from the RPC layer. In this case, we will need a ContainerToken for each container.
> - Do explicit authorization checks without relying on getting ContainerID from the RPC.
> One ContainerToken per container is a serious restriction. We anyways want to be able to use application-ACLS to, say, stop containers owned by others. So I am going to take the later route of explicit checks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3256) Authorization checks needed for
AM->NM protocol
Posted by "Vinod Kumar Vavilapalli (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3256:
-----------------------------------------------
Status: Open (was: Patch Available)
> Authorization checks needed for AM->NM protocol
> -----------------------------------------------
>
> Key: MAPREDUCE-3256
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3256
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: applicationmaster, mrv2, nodemanager, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3256-20111028.1.txt, MAPREDUCE-3256-20111028.2.txt, MAPREDUCE-3256-20111028.2_same, MAPREDUCE-3256-20111029.txt
>
>
> We already authenticate requests to NM from any AM. We also need to authorize the requests, otherwise a rogue AM, *but with proper tokens and thus authenticated to talk to NM*, could either launch or kill a container with different ContainerID. We have two options:
> - Remove the explicit passing of the ContainerId as part of the API and instead get it from the RPC layer. In this case, we will need a ContainerToken for each container.
> - Do explicit authorization checks without relying on getting ContainerID from the RPC.
> One ContainerToken per container is a serious restriction. We anyways want to be able to use application-ACLS to, say, stop containers owned by others. So I am going to take the later route of explicit checks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3256) Authorization checks needed for
AM->NM protocol
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139253#comment-13139253 ]
Hudson commented on MAPREDUCE-3256:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk #847 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/847/])
MAPREDUCE-3256. Added authorization checks for the protocol between NodeManager and ApplicationMaster. Contributed by Vinod K V.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1194850
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/launcher/ContainerLauncherImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestContainerLauncher.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestFail.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/Resource.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ResourcePBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/ContainerTokenIdentifier.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/ContainerTokenSelector.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/BuilderUtils.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/security/ContainerTokenSecretManager.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/ContainerManagerImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/security/LocalizerTokenSelector.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/amlauncher/AMLauncher.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationMasterLauncher.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/TestContainerManagerSecurity.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/TestContainerTokenSecretManager.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/resources/log4j.properties
> Authorization checks needed for AM->NM protocol
> -----------------------------------------------
>
> Key: MAPREDUCE-3256
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3256
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: applicationmaster, mrv2, nodemanager, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3256-20111028.1.txt, MAPREDUCE-3256-20111028.2.txt, MAPREDUCE-3256-20111028.2_same, MAPREDUCE-3256-20111029.1.txt, MAPREDUCE-3256-20111029.txt
>
>
> We already authenticate requests to NM from any AM. We also need to authorize the requests, otherwise a rogue AM, *but with proper tokens and thus authenticated to talk to NM*, could either launch or kill a container with different ContainerID. We have two options:
> - Remove the explicit passing of the ContainerId as part of the API and instead get it from the RPC layer. In this case, we will need a ContainerToken for each container.
> - Do explicit authorization checks without relying on getting ContainerID from the RPC.
> One ContainerToken per container is a serious restriction. We anyways want to be able to use application-ACLS to, say, stop containers owned by others. So I am going to take the later route of explicit checks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3256) Authorization checks needed for
AM->NM protocol
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139164#comment-13139164 ]
Hudson commented on MAPREDUCE-3256:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk-Commit #1271 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1271/])
MAPREDUCE-3256. Added authorization checks for the protocol between NodeManager and ApplicationMaster. Contributed by Vinod K V.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1194850
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/launcher/ContainerLauncherImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestContainerLauncher.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestFail.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/Resource.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ResourcePBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/ContainerTokenIdentifier.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/ContainerTokenSelector.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/BuilderUtils.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/security/ContainerTokenSecretManager.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/ContainerManagerImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/security/LocalizerTokenSelector.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/amlauncher/AMLauncher.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationMasterLauncher.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/TestContainerManagerSecurity.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/TestContainerTokenSecretManager.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/resources/log4j.properties
> Authorization checks needed for AM->NM protocol
> -----------------------------------------------
>
> Key: MAPREDUCE-3256
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3256
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: applicationmaster, mrv2, nodemanager, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3256-20111028.1.txt, MAPREDUCE-3256-20111028.2.txt, MAPREDUCE-3256-20111028.2_same, MAPREDUCE-3256-20111029.1.txt, MAPREDUCE-3256-20111029.txt
>
>
> We already authenticate requests to NM from any AM. We also need to authorize the requests, otherwise a rogue AM, *but with proper tokens and thus authenticated to talk to NM*, could either launch or kill a container with different ContainerID. We have two options:
> - Remove the explicit passing of the ContainerId as part of the API and instead get it from the RPC layer. In this case, we will need a ContainerToken for each container.
> - Do explicit authorization checks without relying on getting ContainerID from the RPC.
> One ContainerToken per container is a serious restriction. We anyways want to be able to use application-ACLS to, say, stop containers owned by others. So I am going to take the later route of explicit checks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3256) Authorization checks needed for
AM->NM protocol
Posted by "Arun C Murthy (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-3256:
-------------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
I just committed this after run a series of java-MR/streaming tests on a secure setup. Thanks Vinod!
> Authorization checks needed for AM->NM protocol
> -----------------------------------------------
>
> Key: MAPREDUCE-3256
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3256
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: applicationmaster, mrv2, nodemanager, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3256-20111028.1.txt, MAPREDUCE-3256-20111028.2.txt, MAPREDUCE-3256-20111028.2_same, MAPREDUCE-3256-20111029.1.txt, MAPREDUCE-3256-20111029.txt
>
>
> We already authenticate requests to NM from any AM. We also need to authorize the requests, otherwise a rogue AM, *but with proper tokens and thus authenticated to talk to NM*, could either launch or kill a container with different ContainerID. We have two options:
> - Remove the explicit passing of the ContainerId as part of the API and instead get it from the RPC layer. In this case, we will need a ContainerToken for each container.
> - Do explicit authorization checks without relying on getting ContainerID from the RPC.
> One ContainerToken per container is a serious restriction. We anyways want to be able to use application-ACLS to, say, stop containers owned by others. So I am going to take the later route of explicit checks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3256) Authorization checks needed for
AM->NM protocol
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13138525#comment-13138525 ]
Hadoop QA commented on MAPREDUCE-3256:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12501322/MAPREDUCE-3256-20111028.1.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 10 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.yarn.server.TestContainerManagerSecurity
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1193//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1193//console
This message is automatically generated.
> Authorization checks needed for AM->NM protocol
> -----------------------------------------------
>
> Key: MAPREDUCE-3256
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3256
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: applicationmaster, mrv2, nodemanager, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3256-20111028.1.txt
>
>
> We already authenticate requests to NM from any AM. We also need to authorize the requests, otherwise a rogue AM, *but with proper tokens and thus authenticated to talk to NM*, could either launch or kill a container with different ContainerID. We have two options:
> - Remove the explicit passing of the ContainerId as part of the API and instead get it from the RPC layer. In this case, we will need a ContainerToken for each container.
> - Do explicit authorization checks without relying on getting ContainerID from the RPC.
> One ContainerToken per container is a serious restriction. We anyways want to be able to use application-ACLS to, say, stop containers owned by others. So I am going to take the later route of explicit checks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3256) Authorization checks needed for
AM->NM protocol
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139225#comment-13139225 ]
Hudson commented on MAPREDUCE-3256:
-----------------------------------
Integrated in Hadoop-Hdfs-0.23-Build #54 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/54/])
Merge -c 1194850 from trunk to branch-0.23 to fix MAPREDUCE-3256.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1194851
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/launcher/ContainerLauncherImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestContainerLauncher.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestFail.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/Resource.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ResourcePBImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/ContainerTokenIdentifier.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/ContainerTokenSelector.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/BuilderUtils.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/security/ContainerTokenSecretManager.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/ContainerManagerImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/security/LocalizerTokenSelector.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/amlauncher/AMLauncher.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationMasterLauncher.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/TestContainerManagerSecurity.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/TestContainerTokenSecretManager.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/resources/log4j.properties
> Authorization checks needed for AM->NM protocol
> -----------------------------------------------
>
> Key: MAPREDUCE-3256
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3256
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: applicationmaster, mrv2, nodemanager, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3256-20111028.1.txt, MAPREDUCE-3256-20111028.2.txt, MAPREDUCE-3256-20111028.2_same, MAPREDUCE-3256-20111029.1.txt, MAPREDUCE-3256-20111029.txt
>
>
> We already authenticate requests to NM from any AM. We also need to authorize the requests, otherwise a rogue AM, *but with proper tokens and thus authenticated to talk to NM*, could either launch or kill a container with different ContainerID. We have two options:
> - Remove the explicit passing of the ContainerId as part of the API and instead get it from the RPC layer. In this case, we will need a ContainerToken for each container.
> - Do explicit authorization checks without relying on getting ContainerID from the RPC.
> One ContainerToken per container is a serious restriction. We anyways want to be able to use application-ACLS to, say, stop containers owned by others. So I am going to take the later route of explicit checks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3256) Authorization checks needed for
AM->NM protocol
Posted by "Vinod Kumar Vavilapalli (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3256:
-----------------------------------------------
Attachment: MAPREDUCE-3256-20111029.txt
This should fix it.
Tested this on a single node secure setup, but I guess that doesn't prove that this patch is better than the previous one.
I don't have access to a bigger cluster. Arun, can you please help with this? Thanks.
> Authorization checks needed for AM->NM protocol
> -----------------------------------------------
>
> Key: MAPREDUCE-3256
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3256
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: applicationmaster, mrv2, nodemanager, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3256-20111028.1.txt, MAPREDUCE-3256-20111028.2.txt, MAPREDUCE-3256-20111028.2_same, MAPREDUCE-3256-20111029.txt
>
>
> We already authenticate requests to NM from any AM. We also need to authorize the requests, otherwise a rogue AM, *but with proper tokens and thus authenticated to talk to NM*, could either launch or kill a container with different ContainerID. We have two options:
> - Remove the explicit passing of the ContainerId as part of the API and instead get it from the RPC layer. In this case, we will need a ContainerToken for each container.
> - Do explicit authorization checks without relying on getting ContainerID from the RPC.
> One ContainerToken per container is a serious restriction. We anyways want to be able to use application-ACLS to, say, stop containers owned by others. So I am going to take the later route of explicit checks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3256) Authorization checks needed for
AM->NM protocol
Posted by "Vinod Kumar Vavilapalli (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3256:
-----------------------------------------------
Status: Patch Available (was: Open)
> Authorization checks needed for AM->NM protocol
> -----------------------------------------------
>
> Key: MAPREDUCE-3256
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3256
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: applicationmaster, mrv2, nodemanager, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3256-20111028.1.txt
>
>
> We already authenticate requests to NM from any AM. We also need to authorize the requests, otherwise a rogue AM, *but with proper tokens and thus authenticated to talk to NM*, could either launch or kill a container with different ContainerID. We have two options:
> - Remove the explicit passing of the ContainerId as part of the API and instead get it from the RPC layer. In this case, we will need a ContainerToken for each container.
> - Do explicit authorization checks without relying on getting ContainerID from the RPC.
> One ContainerToken per container is a serious restriction. We anyways want to be able to use application-ACLS to, say, stop containers owned by others. So I am going to take the later route of explicit checks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3256) Authorization checks needed for
AM->NM protocol
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139167#comment-13139167 ]
Hudson commented on MAPREDUCE-3256:
-----------------------------------
Integrated in Hadoop-Hdfs-0.23-Commit #102 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/102/])
Merge -c 1194850 from trunk to branch-0.23 to fix MAPREDUCE-3256.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1194851
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/launcher/ContainerLauncherImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestContainerLauncher.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestFail.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/Resource.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ResourcePBImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/ContainerTokenIdentifier.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/ContainerTokenSelector.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/BuilderUtils.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/security/ContainerTokenSecretManager.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/ContainerManagerImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/security/LocalizerTokenSelector.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/amlauncher/AMLauncher.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationMasterLauncher.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/TestContainerManagerSecurity.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/TestContainerTokenSecretManager.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/resources/log4j.properties
> Authorization checks needed for AM->NM protocol
> -----------------------------------------------
>
> Key: MAPREDUCE-3256
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3256
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: applicationmaster, mrv2, nodemanager, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3256-20111028.1.txt, MAPREDUCE-3256-20111028.2.txt, MAPREDUCE-3256-20111028.2_same, MAPREDUCE-3256-20111029.1.txt, MAPREDUCE-3256-20111029.txt
>
>
> We already authenticate requests to NM from any AM. We also need to authorize the requests, otherwise a rogue AM, *but with proper tokens and thus authenticated to talk to NM*, could either launch or kill a container with different ContainerID. We have two options:
> - Remove the explicit passing of the ContainerId as part of the API and instead get it from the RPC layer. In this case, we will need a ContainerToken for each container.
> - Do explicit authorization checks without relying on getting ContainerID from the RPC.
> One ContainerToken per container is a serious restriction. We anyways want to be able to use application-ACLS to, say, stop containers owned by others. So I am going to take the later route of explicit checks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3256) Authorization checks needed for
AM->NM protocol
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13138788#comment-13138788 ]
Hadoop QA commented on MAPREDUCE-3256:
--------------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12501363/MAPREDUCE-3256-20111028.2_same
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 10 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1197//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1197//console
This message is automatically generated.
> Authorization checks needed for AM->NM protocol
> -----------------------------------------------
>
> Key: MAPREDUCE-3256
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3256
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: applicationmaster, mrv2, nodemanager, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3256-20111028.1.txt, MAPREDUCE-3256-20111028.2.txt, MAPREDUCE-3256-20111028.2_same
>
>
> We already authenticate requests to NM from any AM. We also need to authorize the requests, otherwise a rogue AM, *but with proper tokens and thus authenticated to talk to NM*, could either launch or kill a container with different ContainerID. We have two options:
> - Remove the explicit passing of the ContainerId as part of the API and instead get it from the RPC layer. In this case, we will need a ContainerToken for each container.
> - Do explicit authorization checks without relying on getting ContainerID from the RPC.
> One ContainerToken per container is a serious restriction. We anyways want to be able to use application-ACLS to, say, stop containers owned by others. So I am going to take the later route of explicit checks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3256) Authorization checks needed for
AM->NM protocol
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139168#comment-13139168 ]
Hudson commented on MAPREDUCE-3256:
-----------------------------------
Integrated in Hadoop-Common-trunk-Commit #1195 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1195/])
MAPREDUCE-3256. Added authorization checks for the protocol between NodeManager and ApplicationMaster. Contributed by Vinod K V.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1194850
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/launcher/ContainerLauncherImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestContainerLauncher.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestFail.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/Resource.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ResourcePBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/ContainerTokenIdentifier.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/ContainerTokenSelector.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/BuilderUtils.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/security/ContainerTokenSecretManager.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/ContainerManagerImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/security/LocalizerTokenSelector.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/amlauncher/AMLauncher.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationMasterLauncher.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/TestContainerManagerSecurity.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/TestContainerTokenSecretManager.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/resources/log4j.properties
> Authorization checks needed for AM->NM protocol
> -----------------------------------------------
>
> Key: MAPREDUCE-3256
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3256
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: applicationmaster, mrv2, nodemanager, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3256-20111028.1.txt, MAPREDUCE-3256-20111028.2.txt, MAPREDUCE-3256-20111028.2_same, MAPREDUCE-3256-20111029.1.txt, MAPREDUCE-3256-20111029.txt
>
>
> We already authenticate requests to NM from any AM. We also need to authorize the requests, otherwise a rogue AM, *but with proper tokens and thus authenticated to talk to NM*, could either launch or kill a container with different ContainerID. We have two options:
> - Remove the explicit passing of the ContainerId as part of the API and instead get it from the RPC layer. In this case, we will need a ContainerToken for each container.
> - Do explicit authorization checks without relying on getting ContainerID from the RPC.
> One ContainerToken per container is a serious restriction. We anyways want to be able to use application-ACLS to, say, stop containers owned by others. So I am going to take the later route of explicit checks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-3256) Authorization checks needed for
AM->NM protocol
Posted by "Vinod Kumar Vavilapalli (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli reassigned MAPREDUCE-3256:
--------------------------------------------------
Assignee: Vinod Kumar Vavilapalli
> Authorization checks needed for AM->NM protocol
> -----------------------------------------------
>
> Key: MAPREDUCE-3256
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3256
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: applicationmaster, nodemanager, resourcemanager, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Priority: Blocker
> Fix For: 0.23.0
>
>
> We already authenticate requests to NM from any AM. We also need to authorize the requests, otherwise a rogue AM, *but with proper tokens and thus authenticated to talk to NM*, could either launch or kill a container with different ContainerID. We have two options:
> - Remove the explicit passing of the ContainerId as part of the API and instead get it from the RPC layer. In this case, we will need a ContainerToken for each container.
> - Do explicit authorization checks without relying on getting ContainerID from the RPC.
> One ContainerToken per container is a serious restriction. We anyways want to be able to use application-ACLS to, say, stop containers owned by others. So I am going to take the later route of explicit checks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3256) Authorization checks needed for
AM->NM protocol
Posted by "Vinod Kumar Vavilapalli (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3256:
-----------------------------------------------
Attachment: MAPREDUCE-3256-20111028.1.txt
Patch to fix the authorization of requests to ContainerManager. Rogue users cannot fake the ContainerID or the Resource anymore.
(Was a fun patch, stepped through most of the RPC code :) ).
> Authorization checks needed for AM->NM protocol
> -----------------------------------------------
>
> Key: MAPREDUCE-3256
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3256
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: applicationmaster, mrv2, nodemanager, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3256-20111028.1.txt
>
>
> We already authenticate requests to NM from any AM. We also need to authorize the requests, otherwise a rogue AM, *but with proper tokens and thus authenticated to talk to NM*, could either launch or kill a container with different ContainerID. We have two options:
> - Remove the explicit passing of the ContainerId as part of the API and instead get it from the RPC layer. In this case, we will need a ContainerToken for each container.
> - Do explicit authorization checks without relying on getting ContainerID from the RPC.
> One ContainerToken per container is a serious restriction. We anyways want to be able to use application-ACLS to, say, stop containers owned by others. So I am going to take the later route of explicit checks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3256) Authorization checks needed for
AM->NM protocol
Posted by "Arun C Murthy (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-3256:
-------------------------------------
Status: Open (was: Patch Available)
Trying jenkins again...
> Authorization checks needed for AM->NM protocol
> -----------------------------------------------
>
> Key: MAPREDUCE-3256
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3256
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: applicationmaster, mrv2, nodemanager, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3256-20111028.1.txt, MAPREDUCE-3256-20111028.2.txt, MAPREDUCE-3256-20111028.2_same
>
>
> We already authenticate requests to NM from any AM. We also need to authorize the requests, otherwise a rogue AM, *but with proper tokens and thus authenticated to talk to NM*, could either launch or kill a container with different ContainerID. We have two options:
> - Remove the explicit passing of the ContainerId as part of the API and instead get it from the RPC layer. In this case, we will need a ContainerToken for each container.
> - Do explicit authorization checks without relying on getting ContainerID from the RPC.
> One ContainerToken per container is a serious restriction. We anyways want to be able to use application-ACLS to, say, stop containers owned by others. So I am going to take the later route of explicit checks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3256) Authorization checks needed for
AM->NM protocol
Posted by "Vinod Kumar Vavilapalli (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3256:
-----------------------------------------------
Status: Patch Available (was: Open)
> Authorization checks needed for AM->NM protocol
> -----------------------------------------------
>
> Key: MAPREDUCE-3256
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3256
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: applicationmaster, mrv2, nodemanager, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3256-20111028.1.txt, MAPREDUCE-3256-20111028.2.txt, MAPREDUCE-3256-20111028.2_same, MAPREDUCE-3256-20111029.1.txt, MAPREDUCE-3256-20111029.txt
>
>
> We already authenticate requests to NM from any AM. We also need to authorize the requests, otherwise a rogue AM, *but with proper tokens and thus authenticated to talk to NM*, could either launch or kill a container with different ContainerID. We have two options:
> - Remove the explicit passing of the ContainerId as part of the API and instead get it from the RPC layer. In this case, we will need a ContainerToken for each container.
> - Do explicit authorization checks without relying on getting ContainerID from the RPC.
> One ContainerToken per container is a serious restriction. We anyways want to be able to use application-ACLS to, say, stop containers owned by others. So I am going to take the later route of explicit checks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira