You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Vinod Kumar Vavilapalli (Created) (JIRA)" <ji...@apache.org> on 2012/02/29 20:27:57 UTC
[jira] [Created] (MAPREDUCE-3940) ContainerTokens should have an
expiry interval
ContainerTokens should have an expiry interval
----------------------------------------------
Key: MAPREDUCE-3940
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2, security
Affects Versions: 0.23.0
Reporter: Vinod Kumar Vavilapalli
Assignee: Vinod Kumar Vavilapalli
Fix For: 0.23.2
- RM should generate the expiry time for a container
- A ContainerToken should have its expire time encoded
- NMs should reject containers with expired tokens.
- Expiry interval for a ContainerToken is same as the expiry interval for a container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3940) ContainerTokens should have an
expiry interval
Posted by "Vinod Kumar Vavilapalli (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3940:
-----------------------------------------------
Resolution: Fixed
Fix Version/s: 0.23.3
Target Version/s: (was: 0.23.3)
Release Note: ContainerTokens now have an expiry interval so that stale tokens cannot be used for launching containers.
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Committed this to trunk, branch-2 and branch-0.23.
Thanks for the bug fixes and the reviews, Sid!
> ContainerTokens should have an expiry interval
> ----------------------------------------------
>
> Key: MAPREDUCE-3940
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Fix For: 0.23.3
>
> Attachments: MAPREDUCE-3940-20120308.txt, MAPREDUCE-3940-20120416.txt, MAPREDUCE-3940-20120425.txt, MAPREDUCE-3940-20120709.txt, MAPREDUCE-3940-20120710.txt, MR3940.txt, MR3940.txt
>
>
> - RM should generate the expiry time for a container
> - A ContainerToken should have its expire time encoded
> - NMs should reject containers with expired tokens.
> - Expiry interval for a ContainerToken is same as the expiry interval for a container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3940) ContainerTokens should have an
expiry interval
Posted by "Vinod Kumar Vavilapalli (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13410911#comment-13410911 ]
Vinod Kumar Vavilapalli commented on MAPREDUCE-3940:
----------------------------------------------------
bq. For consistency with other managers, would it make more sense for the expiration check to be in the secret manager, and for it to throw an InvalidToken exception?
Unfortunately that can't be. That was how the earlier versions of the patch were implemented but as Sid rightly caught a (big) bug above, that would make us use the same expiry checks for stopContainer() and getContainerStatus() calls as well, which is wrong - essentially nobody would be able to stop containers or get statuses.
> ContainerTokens should have an expiry interval
> ----------------------------------------------
>
> Key: MAPREDUCE-3940
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Attachments: MAPREDUCE-3940-20120308.txt, MAPREDUCE-3940-20120416.txt, MAPREDUCE-3940-20120425.txt, MAPREDUCE-3940-20120709.txt, MAPREDUCE-3940-20120710.txt, MR3940.txt, MR3940.txt
>
>
> - RM should generate the expiry time for a container
> - A ContainerToken should have its expire time encoded
> - NMs should reject containers with expired tokens.
> - Expiry interval for a ContainerToken is same as the expiry interval for a container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3940) ContainerTokens should have an
expiry interval
Posted by "Daryn Sharp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13411503#comment-13411503 ]
Daryn Sharp commented on MAPREDUCE-3940:
----------------------------------------
True, the expiry check can't be universal for all operations. An expiry check in the secret manager wouldn't necessary have to be automatically invoked during token validation at the RPC level. It could just be a method that is called when validating the token for a start container. It's not a big deal, but having the expiry validation outside the secret manager seems a bit odd since the expiry is an implementation detail of the secret manager.
After more throughly reviewing the code, it doesn't appear the tokens expire after the container completes? More importantly, the token doesn't appear to become invalid for launching a container after the token has already been used to launch a container?
> ContainerTokens should have an expiry interval
> ----------------------------------------------
>
> Key: MAPREDUCE-3940
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Fix For: 0.23.3
>
> Attachments: MAPREDUCE-3940-20120308.txt, MAPREDUCE-3940-20120416.txt, MAPREDUCE-3940-20120425.txt, MAPREDUCE-3940-20120709.txt, MAPREDUCE-3940-20120710.txt, MR3940.txt, MR3940.txt
>
>
> - RM should generate the expiry time for a container
> - A ContainerToken should have its expire time encoded
> - NMs should reject containers with expired tokens.
> - Expiry interval for a ContainerToken is same as the expiry interval for a container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3940) ContainerTokens should have an
expiry interval
Posted by "Vinod Kumar Vavilapalli (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3940:
-----------------------------------------------
Status: Patch Available (was: Open)
> ContainerTokens should have an expiry interval
> ----------------------------------------------
>
> Key: MAPREDUCE-3940
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Attachments: MAPREDUCE-3940-20120308.txt, MAPREDUCE-3940-20120416.txt, MAPREDUCE-3940-20120425.txt, MAPREDUCE-3940-20120709.txt, MR3940.txt, MR3940.txt
>
>
> - RM should generate the expiry time for a container
> - A ContainerToken should have its expire time encoded
> - NMs should reject containers with expired tokens.
> - Expiry interval for a ContainerToken is same as the expiry interval for a container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3940) ContainerTokens should have an
expiry interval
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13274775#comment-13274775 ]
Hadoop QA commented on MAPREDUCE-3940:
--------------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12526779/MR3940.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 7 new or modified test files.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2387//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2387//console
This message is automatically generated.
> ContainerTokens should have an expiry interval
> ----------------------------------------------
>
> Key: MAPREDUCE-3940
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Attachments: MAPREDUCE-3940-20120308.txt, MAPREDUCE-3940-20120416.txt, MAPREDUCE-3940-20120425.txt, MR3940.txt, MR3940.txt
>
>
> - RM should generate the expiry time for a container
> - A ContainerToken should have its expire time encoded
> - NMs should reject containers with expired tokens.
> - Expiry interval for a ContainerToken is same as the expiry interval for a container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3940) ContainerTokens should have an
expiry interval
Posted by "Siddharth Seth (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13257097#comment-13257097 ]
Siddharth Seth commented on MAPREDUCE-3940:
-------------------------------------------
One issue with the patch...
The MR AM will end up seing some harmless exceptions when invoking stopContainer (similarly for getContainerStatus invocations) for any container which runs beyond the configured limit (default 10 minutes). These containers will however eventually end up being killed via the RM - once the AM releases them.
> ContainerTokens should have an expiry interval
> ----------------------------------------------
>
> Key: MAPREDUCE-3940
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Fix For: 0.23.2
>
> Attachments: MAPREDUCE-3940-20120308.txt, MAPREDUCE-3940-20120416.txt
>
>
> - RM should generate the expiry time for a container
> - A ContainerToken should have its expire time encoded
> - NMs should reject containers with expired tokens.
> - Expiry interval for a ContainerToken is same as the expiry interval for a container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3940) ContainerTokens should have an
expiry interval
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13410753#comment-13410753 ]
Hadoop QA commented on MAPREDUCE-3940:
--------------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12535903/MAPREDUCE-3940-20120710.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 7 new or modified test files.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2567//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2567//console
This message is automatically generated.
> ContainerTokens should have an expiry interval
> ----------------------------------------------
>
> Key: MAPREDUCE-3940
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Attachments: MAPREDUCE-3940-20120308.txt, MAPREDUCE-3940-20120416.txt, MAPREDUCE-3940-20120425.txt, MAPREDUCE-3940-20120709.txt, MAPREDUCE-3940-20120710.txt, MR3940.txt, MR3940.txt
>
>
> - RM should generate the expiry time for a container
> - A ContainerToken should have its expire time encoded
> - NMs should reject containers with expired tokens.
> - Expiry interval for a ContainerToken is same as the expiry interval for a container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3940) ContainerTokens should have an
expiry interval
Posted by "Vinod Kumar Vavilapalli (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3940:
-----------------------------------------------
Attachment: MAPREDUCE-3940-20120308.txt
Attaching patch that adds expiry time to container-token-ids and makes NM reject containers with expired tokens.
Added a unit test to exactly validate the above.
> ContainerTokens should have an expiry interval
> ----------------------------------------------
>
> Key: MAPREDUCE-3940
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Fix For: 0.23.2
>
> Attachments: MAPREDUCE-3940-20120308.txt
>
>
> - RM should generate the expiry time for a container
> - A ContainerToken should have its expire time encoded
> - NMs should reject containers with expired tokens.
> - Expiry interval for a ContainerToken is same as the expiry interval for a container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3940) ContainerTokens should have an
expiry interval
Posted by "Vinod Kumar Vavilapalli (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3940:
-----------------------------------------------
Attachment: MAPREDUCE-3940-20120416.txt
Patch that applies over latest trunk.
> ContainerTokens should have an expiry interval
> ----------------------------------------------
>
> Key: MAPREDUCE-3940
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Fix For: 0.23.2
>
> Attachments: MAPREDUCE-3940-20120308.txt, MAPREDUCE-3940-20120416.txt
>
>
> - RM should generate the expiry time for a container
> - A ContainerToken should have its expire time encoded
> - NMs should reject containers with expired tokens.
> - Expiry interval for a ContainerToken is same as the expiry interval for a container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3940) ContainerTokens should have an
expiry interval
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13261977#comment-13261977 ]
Hadoop QA commented on MAPREDUCE-3940:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12524315/MAPREDUCE-3940-20120425.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 7 new or modified test files.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.yarn.server.TestContainerManagerSecurity
org.apache.hadoop.yarn.server.resourcemanager.security.TestApplicationTokens
org.apache.hadoop.yarn.server.resourcemanager.TestClientRMService
org.apache.hadoop.yarn.server.resourcemanager.resourcetracker.TestNMExpiry
org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization
org.apache.hadoop.yarn.server.resourcemanager.TestApplicationACLs
org.apache.hadoop.mapred.TestClientRedirect
org.apache.hadoop.mapreduce.TestYarnClientProtocolProvider
org.apache.hadoop.mapreduce.security.TestJHSSecurity
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2307//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2307//console
This message is automatically generated.
> ContainerTokens should have an expiry interval
> ----------------------------------------------
>
> Key: MAPREDUCE-3940
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Fix For: 0.23.2
>
> Attachments: MAPREDUCE-3940-20120308.txt, MAPREDUCE-3940-20120416.txt, MAPREDUCE-3940-20120425.txt
>
>
> - RM should generate the expiry time for a container
> - A ContainerToken should have its expire time encoded
> - NMs should reject containers with expired tokens.
> - Expiry interval for a ContainerToken is same as the expiry interval for a container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (MAPREDUCE-3940) ContainerTokens should have an
expiry interval
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy closed MAPREDUCE-3940.
------------------------------------
> ContainerTokens should have an expiry interval
> ----------------------------------------------
>
> Key: MAPREDUCE-3940
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Fix For: 0.23.3, 2.0.2-alpha
>
> Attachments: MAPREDUCE-3940-20120308.txt, MAPREDUCE-3940-20120416.txt, MAPREDUCE-3940-20120425.txt, MAPREDUCE-3940-20120709.txt, MAPREDUCE-3940-20120710.txt, MR3940.txt, MR3940.txt
>
>
> - RM should generate the expiry time for a container
> - A ContainerToken should have its expire time encoded
> - NMs should reject containers with expired tokens.
> - Expiry interval for a ContainerToken is same as the expiry interval for a container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3940) ContainerTokens should have an
expiry interval
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13411036#comment-13411036 ]
Hudson commented on MAPREDUCE-3940:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk-Commit #2511 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2511/])
MAPREDUCE-3940. ContainerTokens should have an expiry interval. Contributed by Siddharth Seth and Vinod Kumar Vavilapalli. (Revision 1359910)
Result = SUCCESS
vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359910
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRMContainerAllocator.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ContainerToken.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/client/ContainerManagerPBClientImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/ContainerTokenIdentifier.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/security/ContainerTokenSecretManager.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/ContainerManagerImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestEventFlow.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/BaseContainerManagerTest.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/TestContainerManager.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmcontainer/ContainerAllocationExpirer.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/resourcetracker/TestNMExpiry.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/resourcetracker/TestRMNMRPCResponseId.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/TestContainerManagerSecurity.java
> ContainerTokens should have an expiry interval
> ----------------------------------------------
>
> Key: MAPREDUCE-3940
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Fix For: 0.23.3
>
> Attachments: MAPREDUCE-3940-20120308.txt, MAPREDUCE-3940-20120416.txt, MAPREDUCE-3940-20120425.txt, MAPREDUCE-3940-20120709.txt, MAPREDUCE-3940-20120710.txt, MR3940.txt, MR3940.txt
>
>
> - RM should generate the expiry time for a container
> - A ContainerToken should have its expire time encoded
> - NMs should reject containers with expired tokens.
> - Expiry interval for a ContainerToken is same as the expiry interval for a container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3940) ContainerTokens should have an
expiry interval
Posted by "Siddharth Seth (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Siddharth Seth updated MAPREDUCE-3940:
--------------------------------------
Attachment: MR3940.txt
Expiry checked only for startContainer calls. Updated the unit test.
> ContainerTokens should have an expiry interval
> ----------------------------------------------
>
> Key: MAPREDUCE-3940
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Attachments: MAPREDUCE-3940-20120308.txt, MAPREDUCE-3940-20120416.txt, MAPREDUCE-3940-20120425.txt, MR3940.txt
>
>
> - RM should generate the expiry time for a container
> - A ContainerToken should have its expire time encoded
> - NMs should reject containers with expired tokens.
> - Expiry interval for a ContainerToken is same as the expiry interval for a container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3940) ContainerTokens should have an
expiry interval
Posted by "Vinod Kumar Vavilapalli (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3940:
-----------------------------------------------
Status: Open (was: Patch Available)
> ContainerTokens should have an expiry interval
> ----------------------------------------------
>
> Key: MAPREDUCE-3940
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Fix For: 0.23.2
>
> Attachments: MAPREDUCE-3940-20120308.txt
>
>
> - RM should generate the expiry time for a container
> - A ContainerToken should have its expire time encoded
> - NMs should reject containers with expired tokens.
> - Expiry interval for a ContainerToken is same as the expiry interval for a container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3940) ContainerTokens should have an
expiry interval
Posted by "Daryn Sharp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13410891#comment-13410891 ]
Daryn Sharp commented on MAPREDUCE-3940:
----------------------------------------
Haven't fully reviewed it, but what I'm seeing is looking good. For consistency with other managers, would it make more sense for the expiration check to be in the secret manager, and for it to throw an {{InvalidToken}} exception?
> ContainerTokens should have an expiry interval
> ----------------------------------------------
>
> Key: MAPREDUCE-3940
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Attachments: MAPREDUCE-3940-20120308.txt, MAPREDUCE-3940-20120416.txt, MAPREDUCE-3940-20120425.txt, MAPREDUCE-3940-20120709.txt, MAPREDUCE-3940-20120710.txt, MR3940.txt, MR3940.txt
>
>
> - RM should generate the expiry time for a container
> - A ContainerToken should have its expire time encoded
> - NMs should reject containers with expired tokens.
> - Expiry interval for a ContainerToken is same as the expiry interval for a container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3940) ContainerTokens should have an
expiry interval
Posted by "Vinod Kumar Vavilapalli (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3940:
-----------------------------------------------
Attachment: MAPREDUCE-3940-20120710.txt
Good catch Sid. This should fix it.
> ContainerTokens should have an expiry interval
> ----------------------------------------------
>
> Key: MAPREDUCE-3940
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Attachments: MAPREDUCE-3940-20120308.txt, MAPREDUCE-3940-20120416.txt, MAPREDUCE-3940-20120425.txt, MAPREDUCE-3940-20120709.txt, MAPREDUCE-3940-20120710.txt, MR3940.txt, MR3940.txt
>
>
> - RM should generate the expiry time for a container
> - A ContainerToken should have its expire time encoded
> - NMs should reject containers with expired tokens.
> - Expiry interval for a ContainerToken is same as the expiry interval for a container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3940) ContainerTokens should have an
expiry interval
Posted by "Siddharth Seth (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13410706#comment-13410706 ]
Siddharth Seth commented on MAPREDUCE-3940:
-------------------------------------------
Needs one change in the FifoScheduler. By returning early on a token creation failure - accounting of 'usedResource' can go wrong. Otherwise looks good.
> ContainerTokens should have an expiry interval
> ----------------------------------------------
>
> Key: MAPREDUCE-3940
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Attachments: MAPREDUCE-3940-20120308.txt, MAPREDUCE-3940-20120416.txt, MAPREDUCE-3940-20120425.txt, MAPREDUCE-3940-20120709.txt, MR3940.txt, MR3940.txt
>
>
> - RM should generate the expiry time for a container
> - A ContainerToken should have its expire time encoded
> - NMs should reject containers with expired tokens.
> - Expiry interval for a ContainerToken is same as the expiry interval for a container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3940) ContainerTokens should have an
expiry interval
Posted by "Vinod Kumar Vavilapalli (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3940:
-----------------------------------------------
Status: Patch Available (was: Open)
> ContainerTokens should have an expiry interval
> ----------------------------------------------
>
> Key: MAPREDUCE-3940
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Attachments: MAPREDUCE-3940-20120308.txt, MAPREDUCE-3940-20120416.txt, MAPREDUCE-3940-20120425.txt, MAPREDUCE-3940-20120709.txt, MAPREDUCE-3940-20120710.txt, MR3940.txt, MR3940.txt
>
>
> - RM should generate the expiry time for a container
> - A ContainerToken should have its expire time encoded
> - NMs should reject containers with expired tokens.
> - Expiry interval for a ContainerToken is same as the expiry interval for a container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3940) ContainerTokens should have an
expiry interval
Posted by "Vinod Kumar Vavilapalli (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3940:
-----------------------------------------------
Status: Open (was: Patch Available)
Thanks for the update, Sid. The fix to move token-expirty to only for startContainer looks good, Thanks.
Will upload a merged patch now.
> ContainerTokens should have an expiry interval
> ----------------------------------------------
>
> Key: MAPREDUCE-3940
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Attachments: MAPREDUCE-3940-20120308.txt, MAPREDUCE-3940-20120416.txt, MAPREDUCE-3940-20120425.txt, MAPREDUCE-3940-20120709.txt, MR3940.txt, MR3940.txt
>
>
> - RM should generate the expiry time for a container
> - A ContainerToken should have its expire time encoded
> - NMs should reject containers with expired tokens.
> - Expiry interval for a ContainerToken is same as the expiry interval for a container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3940) ContainerTokens should have an
expiry interval
Posted by "Vinod Kumar Vavilapalli (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3940:
-----------------------------------------------
Status: Patch Available (was: Open)
> ContainerTokens should have an expiry interval
> ----------------------------------------------
>
> Key: MAPREDUCE-3940
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Fix For: 0.23.2
>
> Attachments: MAPREDUCE-3940-20120308.txt
>
>
> - RM should generate the expiry time for a container
> - A ContainerToken should have its expire time encoded
> - NMs should reject containers with expired tokens.
> - Expiry interval for a ContainerToken is same as the expiry interval for a container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3940) ContainerTokens should have an
expiry interval
Posted by "Siddharth Seth (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Siddharth Seth updated MAPREDUCE-3940:
--------------------------------------
Attachment: MR3940.txt
Upmerged to apply to trunk.
> ContainerTokens should have an expiry interval
> ----------------------------------------------
>
> Key: MAPREDUCE-3940
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Attachments: MAPREDUCE-3940-20120308.txt, MAPREDUCE-3940-20120416.txt, MAPREDUCE-3940-20120425.txt, MR3940.txt, MR3940.txt
>
>
> - RM should generate the expiry time for a container
> - A ContainerToken should have its expire time encoded
> - NMs should reject containers with expired tokens.
> - Expiry interval for a ContainerToken is same as the expiry interval for a container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3940) ContainerTokens should have an
expiry interval
Posted by "Siddharth Seth (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Siddharth Seth updated MAPREDUCE-3940:
--------------------------------------
Status: Patch Available (was: Open)
> ContainerTokens should have an expiry interval
> ----------------------------------------------
>
> Key: MAPREDUCE-3940
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Attachments: MAPREDUCE-3940-20120308.txt, MAPREDUCE-3940-20120416.txt, MAPREDUCE-3940-20120425.txt, MR3940.txt
>
>
> - RM should generate the expiry time for a container
> - A ContainerToken should have its expire time encoded
> - NMs should reject containers with expired tokens.
> - Expiry interval for a ContainerToken is same as the expiry interval for a container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3940) ContainerTokens should have an
expiry interval
Posted by "Siddharth Seth (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Siddharth Seth updated MAPREDUCE-3940:
--------------------------------------
Status: Patch Available (was: Open)
> ContainerTokens should have an expiry interval
> ----------------------------------------------
>
> Key: MAPREDUCE-3940
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Attachments: MAPREDUCE-3940-20120308.txt, MAPREDUCE-3940-20120416.txt, MAPREDUCE-3940-20120425.txt, MR3940.txt, MR3940.txt
>
>
> - RM should generate the expiry time for a container
> - A ContainerToken should have its expire time encoded
> - NMs should reject containers with expired tokens.
> - Expiry interval for a ContainerToken is same as the expiry interval for a container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3940) ContainerTokens should have an
expiry interval
Posted by "Siddharth Seth (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Siddharth Seth updated MAPREDUCE-3940:
--------------------------------------
Fix Version/s: (was: 0.23.2)
Target Version/s: 0.23.3
Status: Open (was: Patch Available)
> ContainerTokens should have an expiry interval
> ----------------------------------------------
>
> Key: MAPREDUCE-3940
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Attachments: MAPREDUCE-3940-20120308.txt, MAPREDUCE-3940-20120416.txt, MAPREDUCE-3940-20120425.txt
>
>
> - RM should generate the expiry time for a container
> - A ContainerToken should have its expire time encoded
> - NMs should reject containers with expired tokens.
> - Expiry interval for a ContainerToken is same as the expiry interval for a container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3940) ContainerTokens should have an
expiry interval
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13411410#comment-13411410 ]
Hudson commented on MAPREDUCE-3940:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk #1100 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1100/])
MAPREDUCE-3940. ContainerTokens should have an expiry interval. Contributed by Siddharth Seth and Vinod Kumar Vavilapalli. (Revision 1359910)
Result = FAILURE
vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359910
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRMContainerAllocator.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ContainerToken.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/client/ContainerManagerPBClientImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/ContainerTokenIdentifier.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/security/ContainerTokenSecretManager.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/ContainerManagerImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestEventFlow.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/BaseContainerManagerTest.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/TestContainerManager.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmcontainer/ContainerAllocationExpirer.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/resourcetracker/TestNMExpiry.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/resourcetracker/TestRMNMRPCResponseId.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/TestContainerManagerSecurity.java
> ContainerTokens should have an expiry interval
> ----------------------------------------------
>
> Key: MAPREDUCE-3940
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Fix For: 0.23.3
>
> Attachments: MAPREDUCE-3940-20120308.txt, MAPREDUCE-3940-20120416.txt, MAPREDUCE-3940-20120425.txt, MAPREDUCE-3940-20120709.txt, MAPREDUCE-3940-20120710.txt, MR3940.txt, MR3940.txt
>
>
> - RM should generate the expiry time for a container
> - A ContainerToken should have its expire time encoded
> - NMs should reject containers with expired tokens.
> - Expiry interval for a ContainerToken is same as the expiry interval for a container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3940) ContainerTokens should have an
expiry interval
Posted by "Vinod Kumar Vavilapalli (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3940:
-----------------------------------------------
Status: Patch Available (was: Open)
> ContainerTokens should have an expiry interval
> ----------------------------------------------
>
> Key: MAPREDUCE-3940
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Fix For: 0.23.2
>
> Attachments: MAPREDUCE-3940-20120308.txt, MAPREDUCE-3940-20120416.txt, MAPREDUCE-3940-20120425.txt
>
>
> - RM should generate the expiry time for a container
> - A ContainerToken should have its expire time encoded
> - NMs should reject containers with expired tokens.
> - Expiry interval for a ContainerToken is same as the expiry interval for a container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3940) ContainerTokens should have an
expiry interval
Posted by "Vinod Kumar Vavilapalli (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3940:
-----------------------------------------------
Issue Type: Sub-task (was: Bug)
Parent: MAPREDUCE-3101
> ContainerTokens should have an expiry interval
> ----------------------------------------------
>
> Key: MAPREDUCE-3940
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Fix For: 0.23.2
>
>
> - RM should generate the expiry time for a container
> - A ContainerToken should have its expire time encoded
> - NMs should reject containers with expired tokens.
> - Expiry interval for a ContainerToken is same as the expiry interval for a container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3940) ContainerTokens should have an
expiry interval
Posted by "Vinod Kumar Vavilapalli (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3940:
-----------------------------------------------
Attachment: MAPREDUCE-3940-20120709.txt
Patch upmerged to trunk.
Also fixed MAPREDUCE-4295 for FifoScheduler.
> ContainerTokens should have an expiry interval
> ----------------------------------------------
>
> Key: MAPREDUCE-3940
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Attachments: MAPREDUCE-3940-20120308.txt, MAPREDUCE-3940-20120416.txt, MAPREDUCE-3940-20120425.txt, MAPREDUCE-3940-20120709.txt, MR3940.txt, MR3940.txt
>
>
> - RM should generate the expiry time for a container
> - A ContainerToken should have its expire time encoded
> - NMs should reject containers with expired tokens.
> - Expiry interval for a ContainerToken is same as the expiry interval for a container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3940) ContainerTokens should have an
expiry interval
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13411524#comment-13411524 ]
Hudson commented on MAPREDUCE-3940:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk #1133 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1133/])
MAPREDUCE-3940. ContainerTokens should have an expiry interval. Contributed by Siddharth Seth and Vinod Kumar Vavilapalli. (Revision 1359910)
Result = SUCCESS
vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359910
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRMContainerAllocator.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ContainerToken.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/client/ContainerManagerPBClientImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/ContainerTokenIdentifier.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/security/ContainerTokenSecretManager.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/ContainerManagerImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestEventFlow.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/BaseContainerManagerTest.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/TestContainerManager.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmcontainer/ContainerAllocationExpirer.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/resourcetracker/TestNMExpiry.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/resourcetracker/TestRMNMRPCResponseId.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/TestContainerManagerSecurity.java
> ContainerTokens should have an expiry interval
> ----------------------------------------------
>
> Key: MAPREDUCE-3940
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Fix For: 0.23.3
>
> Attachments: MAPREDUCE-3940-20120308.txt, MAPREDUCE-3940-20120416.txt, MAPREDUCE-3940-20120425.txt, MAPREDUCE-3940-20120709.txt, MAPREDUCE-3940-20120710.txt, MR3940.txt, MR3940.txt
>
>
> - RM should generate the expiry time for a container
> - A ContainerToken should have its expire time encoded
> - NMs should reject containers with expired tokens.
> - Expiry interval for a ContainerToken is same as the expiry interval for a container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3940) ContainerTokens should have an
expiry interval
Posted by "Vinod Kumar Vavilapalli (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3940:
-----------------------------------------------
Attachment: MAPREDUCE-3940-20120425.txt
bq. The MR AM will end up seing some harmless exceptions when invoking stopContainer
Ths issue of using ContainerTokens with stopContainer() and getContainerStatus() is known, but I did miss the MR AM getting exceptions. We will need to fix circumventing the verification logic of ContainerTokens for the stopContainer() and getContainerStatus() APIs, will open a separate ticket. Short term, we can get this into trunk and merge it into 23 branches after the followup ticket is fixed.
Updating the patch to be applicable against later trunk.
> ContainerTokens should have an expiry interval
> ----------------------------------------------
>
> Key: MAPREDUCE-3940
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Fix For: 0.23.2
>
> Attachments: MAPREDUCE-3940-20120308.txt, MAPREDUCE-3940-20120416.txt, MAPREDUCE-3940-20120425.txt
>
>
> - RM should generate the expiry time for a container
> - A ContainerToken should have its expire time encoded
> - NMs should reject containers with expired tokens.
> - Expiry interval for a ContainerToken is same as the expiry interval for a container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3940) ContainerTokens should have an
expiry interval
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13411478#comment-13411478 ]
Hudson commented on MAPREDUCE-3940:
-----------------------------------
Integrated in Hadoop-Hdfs-0.23-Build #310 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/310/])
MAPREDUCE-3940. ContainerTokens should have an expiry interval. Contributed by Siddharth Seth and Vinod Kumar Vavilapalli.
svn merge -c 1359910 --ignore-ancestry ../../trunk/ (Revision 1359913)
Result = UNSTABLE
vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359913
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRMContainerAllocator.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ContainerToken.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/client/ContainerManagerPBClientImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/ContainerTokenIdentifier.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/security/ContainerTokenSecretManager.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/ContainerManagerImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestEventFlow.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/BaseContainerManagerTest.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/TestContainerManager.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmcontainer/ContainerAllocationExpirer.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/resourcetracker/TestNMExpiry.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/resourcetracker/TestRMNMRPCResponseId.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/TestContainerManagerSecurity.java
> ContainerTokens should have an expiry interval
> ----------------------------------------------
>
> Key: MAPREDUCE-3940
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Fix For: 0.23.3
>
> Attachments: MAPREDUCE-3940-20120308.txt, MAPREDUCE-3940-20120416.txt, MAPREDUCE-3940-20120425.txt, MAPREDUCE-3940-20120709.txt, MAPREDUCE-3940-20120710.txt, MR3940.txt, MR3940.txt
>
>
> - RM should generate the expiry time for a container
> - A ContainerToken should have its expire time encoded
> - NMs should reject containers with expired tokens.
> - Expiry interval for a ContainerToken is same as the expiry interval for a container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3940) ContainerTokens should have an
expiry interval
Posted by "Daryn Sharp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13286752#comment-13286752 ]
Daryn Sharp commented on MAPREDUCE-3940:
----------------------------------------
A few questions/concerns:
# The {{ContainerTokenSecretManager}} appears to be using a hardcoded secret of {{mySecretKey}}?
# Secret managers usually handle the expiration internally to prevent tampering, but the token ident includes the expiry. Combined with the prior point, is it possible to fabricate tokens for any host with any expiration?
# The secret manager usually validates the token & expiration, but here it appears the container manager itself is trying to do it? Does this mean there's no SASL level token check occurring?
# The UGI is the container id instead of the job's submitter?
# The schedulers (fifo & leaf) and secret manager interaction seem inconsistent with the other implementations. {{Token(ident, secretManager)}} seems to be the preferred way to create tokens.
> ContainerTokens should have an expiry interval
> ----------------------------------------------
>
> Key: MAPREDUCE-3940
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Attachments: MAPREDUCE-3940-20120308.txt, MAPREDUCE-3940-20120416.txt, MAPREDUCE-3940-20120425.txt, MR3940.txt, MR3940.txt
>
>
> - RM should generate the expiry time for a container
> - A ContainerToken should have its expire time encoded
> - NMs should reject containers with expired tokens.
> - Expiry interval for a ContainerToken is same as the expiry interval for a container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3940) ContainerTokens should have an
expiry interval
Posted by "Siddharth Seth (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Siddharth Seth updated MAPREDUCE-3940:
--------------------------------------
Status: Open (was: Patch Available)
> ContainerTokens should have an expiry interval
> ----------------------------------------------
>
> Key: MAPREDUCE-3940
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Attachments: MAPREDUCE-3940-20120308.txt, MAPREDUCE-3940-20120416.txt, MAPREDUCE-3940-20120425.txt, MR3940.txt
>
>
> - RM should generate the expiry time for a container
> - A ContainerToken should have its expire time encoded
> - NMs should reject containers with expired tokens.
> - Expiry interval for a ContainerToken is same as the expiry interval for a container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3940) ContainerTokens should have an
expiry interval
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13264141#comment-13264141 ]
Hadoop QA commented on MAPREDUCE-3940:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12524945/MR3940.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 7 new or modified test files.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.mapreduce.TestYarnClientProtocolProvider
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2327//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2327//console
This message is automatically generated.
> ContainerTokens should have an expiry interval
> ----------------------------------------------
>
> Key: MAPREDUCE-3940
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Attachments: MAPREDUCE-3940-20120308.txt, MAPREDUCE-3940-20120416.txt, MAPREDUCE-3940-20120425.txt, MR3940.txt
>
>
> - RM should generate the expiry time for a container
> - A ContainerToken should have its expire time encoded
> - NMs should reject containers with expired tokens.
> - Expiry interval for a ContainerToken is same as the expiry interval for a container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3940) ContainerTokens should have an
expiry interval
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13410183#comment-13410183 ]
Hadoop QA commented on MAPREDUCE-3940:
--------------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12535744/MAPREDUCE-3940-20120709.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 7 new or modified test files.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2563//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2563//console
This message is automatically generated.
> ContainerTokens should have an expiry interval
> ----------------------------------------------
>
> Key: MAPREDUCE-3940
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Attachments: MAPREDUCE-3940-20120308.txt, MAPREDUCE-3940-20120416.txt, MAPREDUCE-3940-20120425.txt, MAPREDUCE-3940-20120709.txt, MR3940.txt, MR3940.txt
>
>
> - RM should generate the expiry time for a container
> - A ContainerToken should have its expire time encoded
> - NMs should reject containers with expired tokens.
> - Expiry interval for a ContainerToken is same as the expiry interval for a container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3940) ContainerTokens should have an
expiry interval
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-3940:
-------------------------------------
Fix Version/s: 2.0.1-alpha
> ContainerTokens should have an expiry interval
> ----------------------------------------------
>
> Key: MAPREDUCE-3940
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Fix For: 0.23.3, 2.0.1-alpha
>
> Attachments: MAPREDUCE-3940-20120308.txt, MAPREDUCE-3940-20120416.txt, MAPREDUCE-3940-20120425.txt, MAPREDUCE-3940-20120709.txt, MAPREDUCE-3940-20120710.txt, MR3940.txt, MR3940.txt
>
>
> - RM should generate the expiry time for a container
> - A ContainerToken should have its expire time encoded
> - NMs should reject containers with expired tokens.
> - Expiry interval for a ContainerToken is same as the expiry interval for a container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3940) ContainerTokens should have an
expiry interval
Posted by "Siddharth Seth (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13410822#comment-13410822 ]
Siddharth Seth commented on MAPREDUCE-3940:
-------------------------------------------
+1. This looks good. Thanks Vinod
> ContainerTokens should have an expiry interval
> ----------------------------------------------
>
> Key: MAPREDUCE-3940
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Attachments: MAPREDUCE-3940-20120308.txt, MAPREDUCE-3940-20120416.txt, MAPREDUCE-3940-20120425.txt, MAPREDUCE-3940-20120709.txt, MAPREDUCE-3940-20120710.txt, MR3940.txt, MR3940.txt
>
>
> - RM should generate the expiry time for a container
> - A ContainerToken should have its expire time encoded
> - NMs should reject containers with expired tokens.
> - Expiry interval for a ContainerToken is same as the expiry interval for a container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3940) ContainerTokens should have an
expiry interval
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13225642#comment-13225642 ]
Hadoop QA commented on MAPREDUCE-3940:
--------------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12517621/MAPREDUCE-3940-20120308.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 21 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2020//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2020//console
This message is automatically generated.
> ContainerTokens should have an expiry interval
> ----------------------------------------------
>
> Key: MAPREDUCE-3940
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Fix For: 0.23.2
>
> Attachments: MAPREDUCE-3940-20120308.txt
>
>
> - RM should generate the expiry time for a container
> - A ContainerToken should have its expire time encoded
> - NMs should reject containers with expired tokens.
> - Expiry interval for a ContainerToken is same as the expiry interval for a container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3940) ContainerTokens should have an
expiry interval
Posted by "Vinod Kumar Vavilapalli (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3940:
-----------------------------------------------
Status: Open (was: Patch Available)
> ContainerTokens should have an expiry interval
> ----------------------------------------------
>
> Key: MAPREDUCE-3940
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Attachments: MAPREDUCE-3940-20120308.txt, MAPREDUCE-3940-20120416.txt, MAPREDUCE-3940-20120425.txt, MAPREDUCE-3940-20120709.txt, MAPREDUCE-3940-20120710.txt, MR3940.txt, MR3940.txt
>
>
> - RM should generate the expiry time for a container
> - A ContainerToken should have its expire time encoded
> - NMs should reject containers with expired tokens.
> - Expiry interval for a ContainerToken is same as the expiry interval for a container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3940) ContainerTokens should have an
expiry interval
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13411030#comment-13411030 ]
Hudson commented on MAPREDUCE-3940:
-----------------------------------
Integrated in Hadoop-Common-trunk-Commit #2444 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2444/])
MAPREDUCE-3940. ContainerTokens should have an expiry interval. Contributed by Siddharth Seth and Vinod Kumar Vavilapalli. (Revision 1359910)
Result = SUCCESS
vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359910
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRMContainerAllocator.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ContainerToken.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/client/ContainerManagerPBClientImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/ContainerTokenIdentifier.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/security/ContainerTokenSecretManager.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/ContainerManagerImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestEventFlow.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/BaseContainerManagerTest.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/TestContainerManager.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmcontainer/ContainerAllocationExpirer.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/resourcetracker/TestNMExpiry.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/resourcetracker/TestRMNMRPCResponseId.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/TestContainerManagerSecurity.java
> ContainerTokens should have an expiry interval
> ----------------------------------------------
>
> Key: MAPREDUCE-3940
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Fix For: 0.23.3
>
> Attachments: MAPREDUCE-3940-20120308.txt, MAPREDUCE-3940-20120416.txt, MAPREDUCE-3940-20120425.txt, MAPREDUCE-3940-20120709.txt, MAPREDUCE-3940-20120710.txt, MR3940.txt, MR3940.txt
>
>
> - RM should generate the expiry time for a container
> - A ContainerToken should have its expire time encoded
> - NMs should reject containers with expired tokens.
> - Expiry interval for a ContainerToken is same as the expiry interval for a container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3940) ContainerTokens should have an
expiry interval
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13411028#comment-13411028 ]
Hudson commented on MAPREDUCE-3940:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #2462 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2462/])
MAPREDUCE-3940. ContainerTokens should have an expiry interval. Contributed by Siddharth Seth and Vinod Kumar Vavilapalli. (Revision 1359910)
Result = FAILURE
vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359910
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRMContainerAllocator.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ContainerToken.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/client/ContainerManagerPBClientImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/ContainerTokenIdentifier.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/security/ContainerTokenSecretManager.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/ContainerManagerImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestEventFlow.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/BaseContainerManagerTest.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/TestContainerManager.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmcontainer/ContainerAllocationExpirer.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/resourcetracker/TestNMExpiry.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/resourcetracker/TestRMNMRPCResponseId.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/TestContainerManagerSecurity.java
> ContainerTokens should have an expiry interval
> ----------------------------------------------
>
> Key: MAPREDUCE-3940
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Fix For: 0.23.3
>
> Attachments: MAPREDUCE-3940-20120308.txt, MAPREDUCE-3940-20120416.txt, MAPREDUCE-3940-20120425.txt, MAPREDUCE-3940-20120709.txt, MAPREDUCE-3940-20120710.txt, MR3940.txt, MR3940.txt
>
>
> - RM should generate the expiry time for a container
> - A ContainerToken should have its expire time encoded
> - NMs should reject containers with expired tokens.
> - Expiry interval for a ContainerToken is same as the expiry interval for a container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3940) ContainerTokens should have an
expiry interval
Posted by "Siddharth Seth (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13288952#comment-13288952 ]
Siddharth Seth commented on MAPREDUCE-3940:
-------------------------------------------
bq. The ContainerTokenSecretManager appears to be using a hardcoded secret of mySecretKey?
That's fixed in MAPREDUCE-3943.
bq. Secret managers usually handle the expiration internally to prevent tampering, but the token ident includes the expiry. Combined with the prior point, is it possible to fabricate tokens for any host with any expiration?
Given the hardcoded secret, that's possible. Again, 3943 fixes this.
bq. The secret manager usually validates the token & expiration, but here it appears the container manager itself is trying to do it? Does this mean there's no SASL level token check occurring?
The secret manager continues to validate parts of token. Expiry is done outside since that's only required for the startContainer call.
bq. The UGI is the container id instead of the job's submitter?
That's used to ensure that the RM did allocate the specific container.
bq. The schedulers (fifo & leaf) and secret manager interaction seem inconsistent with the other implementations. Token(ident, secretManager) seems to be the preferred way to create tokens.
The token identifier and signature are generated by the RM - serialized using PB and then uesd by the AM to communicate with the node manager. The AM generates the Token from the PB message. The RM doesn't really need to create it.
> ContainerTokens should have an expiry interval
> ----------------------------------------------
>
> Key: MAPREDUCE-3940
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Attachments: MAPREDUCE-3940-20120308.txt, MAPREDUCE-3940-20120416.txt, MAPREDUCE-3940-20120425.txt, MR3940.txt, MR3940.txt
>
>
> - RM should generate the expiry time for a container
> - A ContainerToken should have its expire time encoded
> - NMs should reject containers with expired tokens.
> - Expiry interval for a ContainerToken is same as the expiry interval for a container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira