You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Vinod Kumar Vavilapalli (Created) (JIRA)" <ji...@apache.org> on 2012/02/29 22:59:58 UTC
[jira] [Created] (MAPREDUCE-3943) RM-NM secret-keys should be
randomly generated and rolled every so often
RM-NM secret-keys should be randomly generated and rolled every so often
------------------------------------------------------------------------
Key: MAPREDUCE-3943
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3943
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2, security
Affects Versions: 0.23.0
Reporter: Vinod Kumar Vavilapalli
Assignee: Vinod Kumar Vavilapalli
Fix For: 0.23.2
- RM should generate the master-key randomly
- The master-key should roll every so often
- NM should remember old expired keys so that already doled out container-requests can be satisfied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3943) RM-NM secret-keys should be
randomly generated and rolled every so often
Posted by "Daryn Sharp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13287418#comment-13287418 ]
Daryn Sharp commented on MAPREDUCE-3943:
----------------------------------------
bq. Having NMs generate keys can get expensive.
Please elaborate? I'm probably naive about a detail, but it seems like a simple operation to generate the key, register it with the RM which can cache it in its secret manager.
If this jira is integrated as-is, won't later changing the pb messages introduce rolling upgrade incompatibilities?
> RM-NM secret-keys should be randomly generated and rolled every so often
> ------------------------------------------------------------------------
>
> Key: MAPREDUCE-3943
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3943
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Attachments: MAPREDUCE-3943-20120416.txt, MR3943.txt, MR3943.txt
>
>
> - RM should generate the master-key randomly
> - The master-key should roll every so often
> - NM should remember old expired keys so that already doled out container-requests can be satisfied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3943) RM-NM secret-keys should be
randomly generated and rolled every so often
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13425469#comment-13425469 ]
Hadoop QA commented on MAPREDUCE-3943:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12538477/MR3943_trunk.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 17 new or modified test files.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 javadoc. The javadoc tool appears to have generated 1 warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
-1 findbugs. The patch appears to introduce 5 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2679//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2679//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2679//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-nodemanager.html
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2679//console
This message is automatically generated.
> RM-NM secret-keys should be randomly generated and rolled every so often
> ------------------------------------------------------------------------
>
> Key: MAPREDUCE-3943
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3943
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Priority: Critical
> Attachments: MAPREDUCE-3943-20120416.txt, MR3943.txt, MR3943.txt, MR3943_branch-23.txt, MR3943_trunk.txt
>
>
> - RM should generate the master-key randomly
> - The master-key should roll every so often
> - NM should remember old expired keys so that already doled out container-requests can be satisfied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3943) RM-NM secret-keys should be
randomly generated and rolled every so often
Posted by "Siddharth Seth (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13264061#comment-13264061 ]
Siddharth Seth commented on MAPREDUCE-3943:
-------------------------------------------
Spoke to Vinod about this jira and MAPREDUCE-3940. Ideally, longer term - there should be client APIs available for stopping contianers and getting status (based on ACLs). That's not part of these jiras - at least for the moment. Doing something like that via NodeManagers would end up with potentially another Token type, or re-purposing the delegation token.
For now, am uploading a couple of patches based off the original patches - which mostly fix the issues. Still to be tested on a cluster though.
> RM-NM secret-keys should be randomly generated and rolled every so often
> ------------------------------------------------------------------------
>
> Key: MAPREDUCE-3943
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3943
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Fix For: 0.23.2
>
> Attachments: MAPREDUCE-3943-20120416.txt
>
>
> - RM should generate the master-key randomly
> - The master-key should roll every so often
> - NM should remember old expired keys so that already doled out container-requests can be satisfied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3943) RM-NM secret-keys should be
randomly generated and rolled every so often
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13425493#comment-13425493 ]
Hadoop QA commented on MAPREDUCE-3943:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12538489/MR3943_trunk.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 17 new or modified test files.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
-1 findbugs. The patch appears to introduce 4 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2680//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2680//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2680//console
This message is automatically generated.
> RM-NM secret-keys should be randomly generated and rolled every so often
> ------------------------------------------------------------------------
>
> Key: MAPREDUCE-3943
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3943
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Priority: Critical
> Attachments: MAPREDUCE-3943-20120416.txt, MR3943.txt, MR3943.txt, MR3943_branch-23.txt, MR3943_branch-23.txt, MR3943_trunk.txt, MR3943_trunk.txt
>
>
> - RM should generate the master-key randomly
> - The master-key should roll every so often
> - NM should remember old expired keys so that already doled out container-requests can be satisfied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3943) RM-NM secret-keys should be
randomly generated and rolled every so often
Posted by "Siddharth Seth (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Siddharth Seth updated MAPREDUCE-3943:
--------------------------------------
Attachment: MR3943_trunk.txt
Updated to fix javadoc warning and a findbugs warning. 4 of them are from the FairScheduler - unrelated to this patch.
> RM-NM secret-keys should be randomly generated and rolled every so often
> ------------------------------------------------------------------------
>
> Key: MAPREDUCE-3943
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3943
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Priority: Critical
> Attachments: MAPREDUCE-3943-20120416.txt, MR3943.txt, MR3943.txt, MR3943_branch-23.txt, MR3943_branch-23.txt, MR3943_trunk.txt, MR3943_trunk.txt
>
>
> - RM should generate the master-key randomly
> - The master-key should roll every so often
> - NM should remember old expired keys so that already doled out container-requests can be satisfied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3943) RM-NM secret-keys should be
randomly generated and rolled every so often
Posted by "Daryn Sharp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13438284#comment-13438284 ]
Daryn Sharp commented on MAPREDUCE-3943:
----------------------------------------
(Thanks to Jason Lowe for collaborating on the review)
> RM-NM secret-keys should be randomly generated and rolled every so often
> ------------------------------------------------------------------------
>
> Key: MAPREDUCE-3943
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3943
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Priority: Critical
> Attachments: MAPREDUCE-3943-20120416.txt, MR3943_branch-23.txt, MR3943_branch-23.txt, MR3943_trunk.txt, MR3943_trunk.txt, MR3943.txt, MR3943.txt
>
>
> - RM should generate the master-key randomly
> - The master-key should roll every so often
> - NM should remember old expired keys so that already doled out container-requests can be satisfied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3943) RM-NM secret-keys should be
randomly generated and rolled every so often
Posted by "Vinod Kumar Vavilapalli (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13412005#comment-13412005 ]
Vinod Kumar Vavilapalli commented on MAPREDUCE-3943:
----------------------------------------------------
Thanks for the reviews, Sid and Daryn. Thanks for the patch update, Sid.
bq. Passing the shared secret keys in "plaintext" in heartbeats is a bit troubling in general.
Note that NM->RM communication channel is kerberos authenticated, so yeah it is still plain-text but on an authenticated channel.
bq. More concerning is the direction of the data flow: RM generates secret and gives it to the NMs. A rogue or compromised NM can intercept a key which I believe can be used to generate tokens for other NMs.
bq. Moving to per node secrets - I believe that can be achieved via the RM as well, instead of having NMs generate the secret, the RM can generate secrets for each NM. The pb message doesn't change in this case.
It seems like it is good to move to per-node secrets. But let's track it separately.
> RM-NM secret-keys should be randomly generated and rolled every so often
> ------------------------------------------------------------------------
>
> Key: MAPREDUCE-3943
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3943
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Attachments: MAPREDUCE-3943-20120416.txt, MR3943.txt, MR3943.txt
>
>
> - RM should generate the master-key randomly
> - The master-key should roll every so often
> - NM should remember old expired keys so that already doled out container-requests can be satisfied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3943) RM-NM secret-keys should be
randomly generated and rolled every so often
Posted by "Vinod Kumar Vavilapalli (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3943:
-----------------------------------------------
Status: Open (was: Patch Available)
Going to update the patch to reflect the dir structure change after YARN-1.
Thanks for the reviews Daryn and Jason, will look at your comments too.
> RM-NM secret-keys should be randomly generated and rolled every so often
> ------------------------------------------------------------------------
>
> Key: MAPREDUCE-3943
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3943
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Priority: Critical
> Attachments: MAPREDUCE-3943-20120416.txt, MR3943_branch-23.txt, MR3943_branch-23.txt, MR3943_trunk.txt, MR3943_trunk.txt, MR3943.txt, MR3943.txt
>
>
> - RM should generate the master-key randomly
> - The master-key should roll every so often
> - NM should remember old expired keys so that already doled out container-requests can be satisfied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3943) RM-NM secret-keys should be
randomly generated and rolled every so often
Posted by "Vinod Kumar Vavilapalli (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3943:
-----------------------------------------------
Issue Type: Sub-task (was: Bug)
Parent: MAPREDUCE-3101
> RM-NM secret-keys should be randomly generated and rolled every so often
> ------------------------------------------------------------------------
>
> Key: MAPREDUCE-3943
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3943
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Fix For: 0.23.2
>
>
> - RM should generate the master-key randomly
> - The master-key should roll every so often
> - NM should remember old expired keys so that already doled out container-requests can be satisfied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3943) RM-NM secret-keys should be
randomly generated and rolled every so often
Posted by "Siddharth Seth (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Siddharth Seth updated MAPREDUCE-3943:
--------------------------------------
Status: Patch Available (was: Open)
> RM-NM secret-keys should be randomly generated and rolled every so often
> ------------------------------------------------------------------------
>
> Key: MAPREDUCE-3943
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3943
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Priority: Critical
> Attachments: MAPREDUCE-3943-20120416.txt, MR3943.txt, MR3943.txt, MR3943_branch-23.txt, MR3943_branch-23.txt, MR3943_trunk.txt, MR3943_trunk.txt
>
>
> - RM should generate the master-key randomly
> - The master-key should roll every so often
> - NM should remember old expired keys so that already doled out container-requests can be satisfied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3943) RM-NM secret-keys should be
randomly generated and rolled every so often
Posted by "Siddharth Seth (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Siddharth Seth updated MAPREDUCE-3943:
--------------------------------------
Status: Open (was: Patch Available)
> RM-NM secret-keys should be randomly generated and rolled every so often
> ------------------------------------------------------------------------
>
> Key: MAPREDUCE-3943
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3943
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Priority: Critical
> Attachments: MAPREDUCE-3943-20120416.txt, MR3943.txt, MR3943.txt, MR3943_branch-23.txt, MR3943_branch-23.txt, MR3943_trunk.txt
>
>
> - RM should generate the master-key randomly
> - The master-key should roll every so often
> - NM should remember old expired keys so that already doled out container-requests can be satisfied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3943) RM-NM secret-keys should be
randomly generated and rolled every so often
Posted by "Daryn Sharp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13438282#comment-13438282 ]
Daryn Sharp commented on MAPREDUCE-3943:
----------------------------------------
*ContainerManagerImpl*
# {{authorizeRequest}} assuming one and only one token is a landmine. Should either throw if more than one, or use the token selector to get the intended one.
# {{isValidStartContainerRequest(tokenId)}} is invoked before the null check for {{tokenId}}
# Container starts verify the token hasn't already been used. Should status & stop maybe verify it _has_ been used, ie. started?
# Should "Attempt to use an old secret key ..." but something more like "Attempt to relaunch a container"?
*ResourceTrackerService*
# Should refs to {{BaseContainerTokenSecretManager}} be {{RMContainerTokenSecretManager}}?
*NMTokenSecretManager*
# {{retrievePassword}} has a fallback to the current master key when it knows the token is invalid. Why not throw InvalidToken exception?
# {{startContainerSuccessful}} is assuming one and only one token. Perhaps pass it the token validated in {{authorizeRequest}}?
# _Important_: {{appFinished}} "forgets" containers already launched, thus allowing tokens to relaunch containers for 48 hours. The token essentially needs to be cancelled after use which probably means remembering them until hard expiration.
*General questions*
In general, it seems like so many objects shouldn't need to hold a reference to the secret managers. Isn't there a context that can hold a ref?
Using events to sync the secret keys between the {{RMContainerSecretManager}} and {{ResourceTrackerService}} (which actually has a ref to the secret manager!) is rather complex. It was difficult to ensure major race conditions don't exist. Would using the read-write lock, instead of sync keyword, in the secret manager for key access have made it simpler?
> RM-NM secret-keys should be randomly generated and rolled every so often
> ------------------------------------------------------------------------
>
> Key: MAPREDUCE-3943
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3943
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Priority: Critical
> Attachments: MAPREDUCE-3943-20120416.txt, MR3943_branch-23.txt, MR3943_branch-23.txt, MR3943_trunk.txt, MR3943_trunk.txt, MR3943.txt, MR3943.txt
>
>
> - RM should generate the master-key randomly
> - The master-key should roll every so often
> - NM should remember old expired keys so that already doled out container-requests can be satisfied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3943) RM-NM secret-keys should be
randomly generated and rolled every so often
Posted by "Siddharth Seth (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13261805#comment-13261805 ]
Siddharth Seth commented on MAPREDUCE-3943:
-------------------------------------------
Looked at the patch. Needs some fixes.
- There's a possibility of an AM getting a container token using the new master key - before the NM receives the updated key. We have to make sure the NMs receive the updated key before they are used.
- The NM updates it's master key each time it receives one on a heartbeat. The key is being sent on each heartbeat - irrespective of whether it's been updated or not - so the NM will end up losing track of the older keys.
- Minor: ContainerTokenIdentifier doesn't need it's methods to be synchronized.
> RM-NM secret-keys should be randomly generated and rolled every so often
> ------------------------------------------------------------------------
>
> Key: MAPREDUCE-3943
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3943
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Fix For: 0.23.2
>
> Attachments: MAPREDUCE-3943-20120416.txt
>
>
> - RM should generate the master-key randomly
> - The master-key should roll every so often
> - NM should remember old expired keys so that already doled out container-requests can be satisfied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3943) RM-NM secret-keys should be
randomly generated and rolled every so often
Posted by "Vinod Kumar Vavilapalli (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3943:
-----------------------------------------------
Issue Type: Bug (was: Sub-task)
Parent: (was: MAPREDUCE-3101)
> RM-NM secret-keys should be randomly generated and rolled every so often
> ------------------------------------------------------------------------
>
> Key: MAPREDUCE-3943
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3943
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Priority: Critical
> Attachments: MAPREDUCE-3943-20120416.txt, MR3943_branch-23.txt, MR3943_branch-23.txt, MR3943_trunk.txt, MR3943_trunk.txt, MR3943.txt, MR3943.txt
>
>
> - RM should generate the master-key randomly
> - The master-key should roll every so often
> - NM should remember old expired keys so that already doled out container-requests can be satisfied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3943) RM-NM secret-keys should be
randomly generated and rolled every so often
Posted by "Vinod Kumar Vavilapalli (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3943:
-----------------------------------------------
Attachment: MAPREDUCE-3943-20120416.txt
Attaching patch that implements the proposal in the description.
The patch is big enough already, and deserves specific tests which I'm planning to do in follow up tickets.
> RM-NM secret-keys should be randomly generated and rolled every so often
> ------------------------------------------------------------------------
>
> Key: MAPREDUCE-3943
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3943
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Fix For: 0.23.2
>
> Attachments: MAPREDUCE-3943-20120416.txt
>
>
> - RM should generate the master-key randomly
> - The master-key should roll every so often
> - NM should remember old expired keys so that already doled out container-requests can be satisfied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3943) RM-NM secret-keys should be
randomly generated and rolled every so often
Posted by "Siddharth Seth (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Siddharth Seth updated MAPREDUCE-3943:
--------------------------------------
Target Version/s: 0.23.3
Fix Version/s: (was: 0.23.2)
> RM-NM secret-keys should be randomly generated and rolled every so often
> ------------------------------------------------------------------------
>
> Key: MAPREDUCE-3943
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3943
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Attachments: MAPREDUCE-3943-20120416.txt, MR3943.txt
>
>
> - RM should generate the master-key randomly
> - The master-key should roll every so often
> - NM should remember old expired keys so that already doled out container-requests can be satisfied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3943) RM-NM secret-keys should be
randomly generated and rolled every so often
Posted by "Siddharth Seth (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Siddharth Seth updated MAPREDUCE-3943:
--------------------------------------
Attachment: MR3943_branch-23.txt
> RM-NM secret-keys should be randomly generated and rolled every so often
> ------------------------------------------------------------------------
>
> Key: MAPREDUCE-3943
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3943
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Priority: Critical
> Attachments: MAPREDUCE-3943-20120416.txt, MR3943.txt, MR3943.txt, MR3943_branch-23.txt, MR3943_branch-23.txt, MR3943_trunk.txt
>
>
> - RM should generate the master-key randomly
> - The master-key should roll every so often
> - NM should remember old expired keys so that already doled out container-requests can be satisfied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3943) RM-NM secret-keys should be
randomly generated and rolled every so often
Posted by "Siddharth Seth (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Siddharth Seth updated MAPREDUCE-3943:
--------------------------------------
Attachment: MR3943_branch-23.txt
Patch for branch-23.
> RM-NM secret-keys should be randomly generated and rolled every so often
> ------------------------------------------------------------------------
>
> Key: MAPREDUCE-3943
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3943
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Priority: Critical
> Attachments: MAPREDUCE-3943-20120416.txt, MR3943.txt, MR3943.txt, MR3943_branch-23.txt, MR3943_trunk.txt
>
>
> - RM should generate the master-key randomly
> - The master-key should roll every so often
> - NM should remember old expired keys so that already doled out container-requests can be satisfied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3943) RM-NM secret-keys should be
randomly generated and rolled every so often
Posted by "Daryn Sharp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271756#comment-13271756 ]
Daryn Sharp commented on MAPREDUCE-3943:
----------------------------------------
Just FYI, the patch doesn't apply.
It feels a bit contorted for the RM to have a pb message with the current and prior key (ie. it's limited), which is the root of the 2X key roll problem. With the patch the way it is, having the RM transmit a single key and the NM remembering N-many keys is probably "less bad"...?
Passing the shared secret keys in "plaintext" in heartbeats is a bit troubling in general. More concerning is the direction of the data flow: RM generates secret and gives it to the NMs. A rogue or compromised NM can intercept a key which I believe can be used to generate tokens for other NMs. If true, doesn't that put the entire cluster at risk?
Conceptually, the RM should simply request a token from the NM and pass the token along to the AM so it can contact the NM. It that's too expensive, it seems like the key exchange should be inverted: NMs generate their own secret, and provide that secret to the RM. A compromised node cannot damage the entire cluster.
> RM-NM secret-keys should be randomly generated and rolled every so often
> ------------------------------------------------------------------------
>
> Key: MAPREDUCE-3943
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3943
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Attachments: MAPREDUCE-3943-20120416.txt, MR3943.txt
>
>
> - RM should generate the master-key randomly
> - The master-key should roll every so often
> - NM should remember old expired keys so that already doled out container-requests can be satisfied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3943) RM-NM secret-keys should be
randomly generated and rolled every so often
Posted by "Siddharth Seth (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13288959#comment-13288959 ]
Siddharth Seth commented on MAPREDUCE-3943:
-------------------------------------------
bq. Having NMs generate keys can get expensive.
Please elaborate? I'm probably naive about a detail, but it seems like a simple operation to generate the key, register it with the RM which can cache it in its secret manager.
That was supposed to read - having NMs generate 'tokens' can get expensive. With the current protocol - the RM sets the allocated containerId as part of the token identifier. The RM would have to inform the NM about the containerId, have it generate the token and then provide it to the AM. That's several heartbeats.
Moving to per node secrets - I believe that can be achieved via the RM as well, instead of having NMs generate the secret, the RM can generate secrets for each NM. The pb message doesn't change in this case.
> RM-NM secret-keys should be randomly generated and rolled every so often
> ------------------------------------------------------------------------
>
> Key: MAPREDUCE-3943
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3943
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Attachments: MAPREDUCE-3943-20120416.txt, MR3943.txt, MR3943.txt
>
>
> - RM should generate the master-key randomly
> - The master-key should roll every so often
> - NM should remember old expired keys so that already doled out container-requests can be satisfied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3943) RM-NM secret-keys should be
randomly generated and rolled every so often
Posted by "Siddharth Seth (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Siddharth Seth updated MAPREDUCE-3943:
--------------------------------------
Attachment: MR3943.txt
Should fix the two issues mentioned previously.
Depends on 3940, and hasn't been tested yet.
stopContainer and getContainerStatus will end up causing exceptions after 2X rolling interval. Have half a patch to fix that - by remembering older container specific tokens on the NodeManager - but not very happy with that approach.
> RM-NM secret-keys should be randomly generated and rolled every so often
> ------------------------------------------------------------------------
>
> Key: MAPREDUCE-3943
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3943
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Attachments: MAPREDUCE-3943-20120416.txt, MR3943.txt
>
>
> - RM should generate the master-key randomly
> - The master-key should roll every so often
> - NM should remember old expired keys so that already doled out container-requests can be satisfied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3943) RM-NM secret-keys should be
randomly generated and rolled every so often
Posted by "Vinod Kumar Vavilapalli (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13438341#comment-13438341 ]
Vinod Kumar Vavilapalli commented on MAPREDUCE-3943:
----------------------------------------------------
bq. It seems like it is good to move to per-node secrets. But let's track it separately.
Filed YARN-35.
> RM-NM secret-keys should be randomly generated and rolled every so often
> ------------------------------------------------------------------------
>
> Key: MAPREDUCE-3943
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3943
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Priority: Critical
> Attachments: MAPREDUCE-3943-20120416.txt, MR3943_branch-23.txt, MR3943_branch-23.txt, MR3943_trunk.txt, MR3943_trunk.txt, MR3943.txt, MR3943.txt
>
>
> - RM should generate the master-key randomly
> - The master-key should roll every so often
> - NM should remember old expired keys so that already doled out container-requests can be satisfied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3943) RM-NM secret-keys should be
randomly generated and rolled every so often
Posted by "Siddharth Seth (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Siddharth Seth updated MAPREDUCE-3943:
--------------------------------------
Attachment: MR3943_trunk.txt
Up-merged patch for trunk. Also ads some securityEnabled checks.
> RM-NM secret-keys should be randomly generated and rolled every so often
> ------------------------------------------------------------------------
>
> Key: MAPREDUCE-3943
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3943
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Priority: Critical
> Attachments: MAPREDUCE-3943-20120416.txt, MR3943.txt, MR3943.txt, MR3943_branch-23.txt, MR3943_trunk.txt
>
>
> - RM should generate the master-key randomly
> - The master-key should roll every so often
> - NM should remember old expired keys so that already doled out container-requests can be satisfied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3943) RM-NM secret-keys should be
randomly generated and rolled every so often
Posted by "Thomas Graves (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Graves updated MAPREDUCE-3943:
-------------------------------------
Priority: Critical (was: Major)
> RM-NM secret-keys should be randomly generated and rolled every so often
> ------------------------------------------------------------------------
>
> Key: MAPREDUCE-3943
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3943
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Priority: Critical
> Attachments: MAPREDUCE-3943-20120416.txt, MR3943.txt, MR3943.txt
>
>
> - RM should generate the master-key randomly
> - The master-key should roll every so often
> - NM should remember old expired keys so that already doled out container-requests can be satisfied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3943) RM-NM secret-keys should be
randomly generated and rolled every so often
Posted by "Siddharth Seth (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13272751#comment-13272751 ]
Siddharth Seth commented on MAPREDUCE-3943:
-------------------------------------------
Thanks for taking a look Daryn.
The patch requires MAPREDUCE-3940 to be applied first (if that still applies).
bq. It feels a bit contorted for the RM to have a pb message with the current and prior key (ie. it's limited), which is the root of the 2X key roll problem. With the patch the way it is, having the RM transmit a single key and the NM remembering N-many keys is probably "less bad"...?
Current and prior keys are only sent during nodeRegistration - in case a node registers in the window where the RM has generated a new key but hasn't started using it yet. nodeHeartbeat calls only send the latest key.
The NM remembers 2 keys - which is what causes the 2x key roll problem. Remembering additional keys on the NM needs to be controlled - older keys can't be used to startContainers. The half patch I mentioned earlier - would get the NM to remember additional keys for containers which have already been started. Will incorporate that into this patch.
bq. Passing the shared secret keys in "plaintext" in heartbeats is a bit troubling in general. More concerning is the direction of the data flow: RM generates secret and gives it to the NMs. A rogue or compromised NM can intercept a key which I believe can be used to generate tokens for other NMs. If true, doesn't that put the entire cluster at risk?
bq. Conceptually, the RM should simply request a token from the NM and pass the token along to the AM so it can contact the NM. It that's too expensive, it seems like the key exchange should be inverted: NMs generate their own secret, and provide that secret to the RM. A compromised node cannot damage the entire cluster.
Having NMs generate keys can get expensive.NMs generate keys and provide them to the RM can be achieved by having the RM generate individual keys per NM. I'd prefer dealing with that in a separate jira - since it's a reasonably independent change.
> RM-NM secret-keys should be randomly generated and rolled every so often
> ------------------------------------------------------------------------
>
> Key: MAPREDUCE-3943
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3943
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Attachments: MAPREDUCE-3943-20120416.txt, MR3943.txt
>
>
> - RM should generate the master-key randomly
> - The master-key should roll every so often
> - NM should remember old expired keys so that already doled out container-requests can be satisfied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3943) RM-NM secret-keys should be
randomly generated and rolled every so often
Posted by "Siddharth Seth (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Siddharth Seth updated MAPREDUCE-3943:
--------------------------------------
Status: Patch Available (was: Open)
> RM-NM secret-keys should be randomly generated and rolled every so often
> ------------------------------------------------------------------------
>
> Key: MAPREDUCE-3943
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3943
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Priority: Critical
> Attachments: MAPREDUCE-3943-20120416.txt, MR3943.txt, MR3943.txt, MR3943_branch-23.txt, MR3943_trunk.txt
>
>
> - RM should generate the master-key randomly
> - The master-key should roll every so often
> - NM should remember old expired keys so that already doled out container-requests can be satisfied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3943) RM-NM secret-keys should be
randomly generated and rolled every so often
Posted by "Siddharth Seth (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Siddharth Seth updated MAPREDUCE-3943:
--------------------------------------
Attachment: MR3943.txt
NM remembers additional secrets to prevent the exceptions on stopContainer/getContainerStatus.
Tested this along with MAPREDUCE-3940 on a secure cluster. Required 3940 to be applied first.
> RM-NM secret-keys should be randomly generated and rolled every so often
> ------------------------------------------------------------------------
>
> Key: MAPREDUCE-3943
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3943
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Attachments: MAPREDUCE-3943-20120416.txt, MR3943.txt, MR3943.txt
>
>
> - RM should generate the master-key randomly
> - The master-key should roll every so often
> - NM should remember old expired keys so that already doled out container-requests can be satisfied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira