You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Daryn Sharp (Created) (JIRA)" <ji...@apache.org> on 2012/02/10 19:56:59 UTC
[jira] [Created] (MAPREDUCE-3849) Change TokenCache's reading of
the binary token file
Change TokenCache's reading of the binary token file
----------------------------------------------------
Key: MAPREDUCE-3849
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3849
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: security
Affects Versions: 0.23.1, 0.24.0
Reporter: Daryn Sharp
Assignee: Daryn Sharp
When obtaining the tokens for a {{FileSystem}}, the {{TokenCache}} will read the binary token file if a token is not already in the {{Credentials}}. However, it will overwrite any existing tokens in the {{Credentials}} with the contents of the binary token file if a single token is missing. This may cause new tokens to be replaced with invalid/cancelled tokens from the binary file. The new tokens will not be canceled, and thus "leak" in the namenode until they expire.
The binary tokens should be merged with, but not replace, existing tokens in the {{Credentials}}.
The code that reads the binary token file is prefaced with:
{code}
//TODO: Need to come up with a better place to put
//this block of code to do with reading the file
{code}
Also, the loading of the binary token file is the only reason that the {{TokenCache}} has to use {{getCanonicalService}}. If this linkage can be broken, then the 1-to-1 filesystem to token service coupling may be removed. And use of {{getCanonicalService}} can be removed in a subsequent jira.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3849) Change TokenCache's reading of
the binary token file
Posted by "Robert Joseph Evans (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207025#comment-13207025 ]
Robert Joseph Evans commented on MAPREDUCE-3849:
------------------------------------------------
The patch looks good. Let me test it. Could you please check to be sure that there is a JIRA for the failing test.
> Change TokenCache's reading of the binary token file
> ----------------------------------------------------
>
> Key: MAPREDUCE-3849
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3849
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: security
> Affects Versions: 0.23.1, 0.24.0
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Attachments: MAPREDUCE-3849.patch
>
>
> When obtaining the tokens for a {{FileSystem}}, the {{TokenCache}} will read the binary token file if a token is not already in the {{Credentials}}. However, it will overwrite any existing tokens in the {{Credentials}} with the contents of the binary token file if a single token is missing. This may cause new tokens to be replaced with invalid/cancelled tokens from the binary file. The new tokens will not be canceled, and thus "leak" in the namenode until they expire.
> The binary tokens should be merged with, but not replace, existing tokens in the {{Credentials}}.
> The code that reads the binary token file is prefaced with:
> {code}
> //TODO: Need to come up with a better place to put
> //this block of code to do with reading the file
> {code}
> Also, the loading of the binary token file is the only reason that the {{TokenCache}} has to use {{getCanonicalService}}. If this linkage can be broken, then the 1-to-1 filesystem to token service coupling may be removed. And use of {{getCanonicalService}} can be removed in a subsequent jira.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3849) Change TokenCache's reading of
the binary token file
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13210249#comment-13210249 ]
Hudson commented on MAPREDUCE-3849:
-----------------------------------
Integrated in Hadoop-Mapreduce-0.23-Build #199 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/199/])
svn merge -c 1245099 from trunk to 0.23 FIXES MAPREDUCE-3849 Change TokenCache's reading of the binary token file (Daryn Sharp via bobby) (Revision 1245102)
Result = FAILURE
bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1245102
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/security/TokenCache.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/security/TestTokenCache.java
> Change TokenCache's reading of the binary token file
> ----------------------------------------------------
>
> Key: MAPREDUCE-3849
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3849
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: security
> Affects Versions: 0.23.1, 0.24.0
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Fix For: 0.23.2
>
> Attachments: MAPREDUCE-3849-2.patch, MAPREDUCE-3849.patch
>
>
> When obtaining the tokens for a {{FileSystem}}, the {{TokenCache}} will read the binary token file if a token is not already in the {{Credentials}}. However, it will overwrite any existing tokens in the {{Credentials}} with the contents of the binary token file if a single token is missing. This may cause new tokens to be replaced with invalid/cancelled tokens from the binary file. The new tokens will not be canceled, and thus "leak" in the namenode until they expire.
> The binary tokens should be merged with, but not replace, existing tokens in the {{Credentials}}.
> The code that reads the binary token file is prefaced with:
> {code}
> //TODO: Need to come up with a better place to put
> //this block of code to do with reading the file
> {code}
> Also, the loading of the binary token file is the only reason that the {{TokenCache}} has to use {{getCanonicalService}}. If this linkage can be broken, then the 1-to-1 filesystem to token service coupling may be removed. And use of {{getCanonicalService}} can be removed in a subsequent jira.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3849) Change TokenCache's reading of
the binary token file
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209545#comment-13209545 ]
Hudson commented on MAPREDUCE-3849:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk-Commit #1809 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1809/])
MAPREDUCE-3849. Change TokenCache's reading of the binary token file (Daryn Sharp via bobby) (Revision 1245099)
Result = SUCCESS
bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1245099
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/security/TokenCache.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/security/TestTokenCache.java
> Change TokenCache's reading of the binary token file
> ----------------------------------------------------
>
> Key: MAPREDUCE-3849
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3849
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: security
> Affects Versions: 0.23.1, 0.24.0
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Attachments: MAPREDUCE-3849-2.patch, MAPREDUCE-3849.patch
>
>
> When obtaining the tokens for a {{FileSystem}}, the {{TokenCache}} will read the binary token file if a token is not already in the {{Credentials}}. However, it will overwrite any existing tokens in the {{Credentials}} with the contents of the binary token file if a single token is missing. This may cause new tokens to be replaced with invalid/cancelled tokens from the binary file. The new tokens will not be canceled, and thus "leak" in the namenode until they expire.
> The binary tokens should be merged with, but not replace, existing tokens in the {{Credentials}}.
> The code that reads the binary token file is prefaced with:
> {code}
> //TODO: Need to come up with a better place to put
> //this block of code to do with reading the file
> {code}
> Also, the loading of the binary token file is the only reason that the {{TokenCache}} has to use {{getCanonicalService}}. If this linkage can be broken, then the 1-to-1 filesystem to token service coupling may be removed. And use of {{getCanonicalService}} can be removed in a subsequent jira.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3849) Change TokenCache's reading of
the binary token file
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205683#comment-13205683 ]
Hadoop QA commented on MAPREDUCE-3849:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12514141/MAPREDUCE-3849.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.yarn.util.TestLinuxResourceCalculatorPlugin
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1837//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1837//console
This message is automatically generated.
> Change TokenCache's reading of the binary token file
> ----------------------------------------------------
>
> Key: MAPREDUCE-3849
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3849
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: security
> Affects Versions: 0.23.1, 0.24.0
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Attachments: MAPREDUCE-3849.patch
>
>
> When obtaining the tokens for a {{FileSystem}}, the {{TokenCache}} will read the binary token file if a token is not already in the {{Credentials}}. However, it will overwrite any existing tokens in the {{Credentials}} with the contents of the binary token file if a single token is missing. This may cause new tokens to be replaced with invalid/cancelled tokens from the binary file. The new tokens will not be canceled, and thus "leak" in the namenode until they expire.
> The binary tokens should be merged with, but not replace, existing tokens in the {{Credentials}}.
> The code that reads the binary token file is prefaced with:
> {code}
> //TODO: Need to come up with a better place to put
> //this block of code to do with reading the file
> {code}
> Also, the loading of the binary token file is the only reason that the {{TokenCache}} has to use {{getCanonicalService}}. If this linkage can be broken, then the 1-to-1 filesystem to token service coupling may be removed. And use of {{getCanonicalService}} can be removed in a subsequent jira.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3849) Change TokenCache's reading of
the binary token file
Posted by "Robert Joseph Evans (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Joseph Evans updated MAPREDUCE-3849:
-------------------------------------------
Resolution: Fixed
Fix Version/s: 0.23.2
Target Version/s: 0.23.0, 0.24.0 (was: 0.24.0, 0.23.0)
Status: Resolved (was: Patch Available)
Thanks Daryn I just checked this into trunk and 0.23.2
> Change TokenCache's reading of the binary token file
> ----------------------------------------------------
>
> Key: MAPREDUCE-3849
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3849
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: security
> Affects Versions: 0.23.1, 0.24.0
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Fix For: 0.23.2
>
> Attachments: MAPREDUCE-3849-2.patch, MAPREDUCE-3849.patch
>
>
> When obtaining the tokens for a {{FileSystem}}, the {{TokenCache}} will read the binary token file if a token is not already in the {{Credentials}}. However, it will overwrite any existing tokens in the {{Credentials}} with the contents of the binary token file if a single token is missing. This may cause new tokens to be replaced with invalid/cancelled tokens from the binary file. The new tokens will not be canceled, and thus "leak" in the namenode until they expire.
> The binary tokens should be merged with, but not replace, existing tokens in the {{Credentials}}.
> The code that reads the binary token file is prefaced with:
> {code}
> //TODO: Need to come up with a better place to put
> //this block of code to do with reading the file
> {code}
> Also, the loading of the binary token file is the only reason that the {{TokenCache}} has to use {{getCanonicalService}}. If this linkage can be broken, then the 1-to-1 filesystem to token service coupling may be removed. And use of {{getCanonicalService}} can be removed in a subsequent jira.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3849) Change TokenCache's reading of
the binary token file
Posted by "Daryn Sharp (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13208608#comment-13208608 ]
Daryn Sharp commented on MAPREDUCE-3849:
----------------------------------------
This was also successfully tested on a secure cluster.
> Change TokenCache's reading of the binary token file
> ----------------------------------------------------
>
> Key: MAPREDUCE-3849
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3849
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: security
> Affects Versions: 0.23.1, 0.24.0
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Attachments: MAPREDUCE-3849-2.patch, MAPREDUCE-3849.patch
>
>
> When obtaining the tokens for a {{FileSystem}}, the {{TokenCache}} will read the binary token file if a token is not already in the {{Credentials}}. However, it will overwrite any existing tokens in the {{Credentials}} with the contents of the binary token file if a single token is missing. This may cause new tokens to be replaced with invalid/cancelled tokens from the binary file. The new tokens will not be canceled, and thus "leak" in the namenode until they expire.
> The binary tokens should be merged with, but not replace, existing tokens in the {{Credentials}}.
> The code that reads the binary token file is prefaced with:
> {code}
> //TODO: Need to come up with a better place to put
> //this block of code to do with reading the file
> {code}
> Also, the loading of the binary token file is the only reason that the {{TokenCache}} has to use {{getCanonicalService}}. If this linkage can be broken, then the 1-to-1 filesystem to token service coupling may be removed. And use of {{getCanonicalService}} can be removed in a subsequent jira.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3849) Change TokenCache's reading of
the binary token file
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209557#comment-13209557 ]
Hudson commented on MAPREDUCE-3849:
-----------------------------------
Integrated in Hadoop-Common-0.23-Commit #559 (See [https://builds.apache.org/job/Hadoop-Common-0.23-Commit/559/])
svn merge -c 1245099 from trunk to 0.23 FIXES MAPREDUCE-3849 Change TokenCache's reading of the binary token file (Daryn Sharp via bobby) (Revision 1245102)
Result = SUCCESS
bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1245102
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/security/TokenCache.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/security/TestTokenCache.java
> Change TokenCache's reading of the binary token file
> ----------------------------------------------------
>
> Key: MAPREDUCE-3849
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3849
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: security
> Affects Versions: 0.23.1, 0.24.0
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Fix For: 0.23.2
>
> Attachments: MAPREDUCE-3849-2.patch, MAPREDUCE-3849.patch
>
>
> When obtaining the tokens for a {{FileSystem}}, the {{TokenCache}} will read the binary token file if a token is not already in the {{Credentials}}. However, it will overwrite any existing tokens in the {{Credentials}} with the contents of the binary token file if a single token is missing. This may cause new tokens to be replaced with invalid/cancelled tokens from the binary file. The new tokens will not be canceled, and thus "leak" in the namenode until they expire.
> The binary tokens should be merged with, but not replace, existing tokens in the {{Credentials}}.
> The code that reads the binary token file is prefaced with:
> {code}
> //TODO: Need to come up with a better place to put
> //this block of code to do with reading the file
> {code}
> Also, the loading of the binary token file is the only reason that the {{TokenCache}} has to use {{getCanonicalService}}. If this linkage can be broken, then the 1-to-1 filesystem to token service coupling may be removed. And use of {{getCanonicalService}} can be removed in a subsequent jira.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3849) Change TokenCache's reading of
the binary token file
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209595#comment-13209595 ]
Hudson commented on MAPREDUCE-3849:
-----------------------------------
Integrated in Hadoop-Mapreduce-0.23-Commit #562 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/562/])
svn merge -c 1245099 from trunk to 0.23 FIXES MAPREDUCE-3849 Change TokenCache's reading of the binary token file (Daryn Sharp via bobby) (Revision 1245102)
Result = ABORTED
bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1245102
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/security/TokenCache.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/security/TestTokenCache.java
> Change TokenCache's reading of the binary token file
> ----------------------------------------------------
>
> Key: MAPREDUCE-3849
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3849
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: security
> Affects Versions: 0.23.1, 0.24.0
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Fix For: 0.23.2
>
> Attachments: MAPREDUCE-3849-2.patch, MAPREDUCE-3849.patch
>
>
> When obtaining the tokens for a {{FileSystem}}, the {{TokenCache}} will read the binary token file if a token is not already in the {{Credentials}}. However, it will overwrite any existing tokens in the {{Credentials}} with the contents of the binary token file if a single token is missing. This may cause new tokens to be replaced with invalid/cancelled tokens from the binary file. The new tokens will not be canceled, and thus "leak" in the namenode until they expire.
> The binary tokens should be merged with, but not replace, existing tokens in the {{Credentials}}.
> The code that reads the binary token file is prefaced with:
> {code}
> //TODO: Need to come up with a better place to put
> //this block of code to do with reading the file
> {code}
> Also, the loading of the binary token file is the only reason that the {{TokenCache}} has to use {{getCanonicalService}}. If this linkage can be broken, then the 1-to-1 filesystem to token service coupling may be removed. And use of {{getCanonicalService}} can be removed in a subsequent jira.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3849) Change TokenCache's reading of
the binary token file
Posted by "Daryn Sharp (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daryn Sharp updated MAPREDUCE-3849:
-----------------------------------
Attachment: MAPREDUCE-3849.patch
> Change TokenCache's reading of the binary token file
> ----------------------------------------------------
>
> Key: MAPREDUCE-3849
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3849
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: security
> Affects Versions: 0.23.1, 0.24.0
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Attachments: MAPREDUCE-3849.patch
>
>
> When obtaining the tokens for a {{FileSystem}}, the {{TokenCache}} will read the binary token file if a token is not already in the {{Credentials}}. However, it will overwrite any existing tokens in the {{Credentials}} with the contents of the binary token file if a single token is missing. This may cause new tokens to be replaced with invalid/cancelled tokens from the binary file. The new tokens will not be canceled, and thus "leak" in the namenode until they expire.
> The binary tokens should be merged with, but not replace, existing tokens in the {{Credentials}}.
> The code that reads the binary token file is prefaced with:
> {code}
> //TODO: Need to come up with a better place to put
> //this block of code to do with reading the file
> {code}
> Also, the loading of the binary token file is the only reason that the {{TokenCache}} has to use {{getCanonicalService}}. If this linkage can be broken, then the 1-to-1 filesystem to token service coupling may be removed. And use of {{getCanonicalService}} can be removed in a subsequent jira.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3849) Change TokenCache's reading of
the binary token file
Posted by "Daryn Sharp (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205760#comment-13205760 ]
Daryn Sharp commented on MAPREDUCE-3849:
----------------------------------------
Failed test is not related to this patch, and it has been failing since yesterday.
> Change TokenCache's reading of the binary token file
> ----------------------------------------------------
>
> Key: MAPREDUCE-3849
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3849
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: security
> Affects Versions: 0.23.1, 0.24.0
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Attachments: MAPREDUCE-3849.patch
>
>
> When obtaining the tokens for a {{FileSystem}}, the {{TokenCache}} will read the binary token file if a token is not already in the {{Credentials}}. However, it will overwrite any existing tokens in the {{Credentials}} with the contents of the binary token file if a single token is missing. This may cause new tokens to be replaced with invalid/cancelled tokens from the binary file. The new tokens will not be canceled, and thus "leak" in the namenode until they expire.
> The binary tokens should be merged with, but not replace, existing tokens in the {{Credentials}}.
> The code that reads the binary token file is prefaced with:
> {code}
> //TODO: Need to come up with a better place to put
> //this block of code to do with reading the file
> {code}
> Also, the loading of the binary token file is the only reason that the {{TokenCache}} has to use {{getCanonicalService}}. If this linkage can be broken, then the 1-to-1 filesystem to token service coupling may be removed. And use of {{getCanonicalService}} can be removed in a subsequent jira.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3849) Change TokenCache's reading of
the binary token file
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13210226#comment-13210226 ]
Hudson commented on MAPREDUCE-3849:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk #958 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/958/])
MAPREDUCE-3849. Change TokenCache's reading of the binary token file (Daryn Sharp via bobby) (Revision 1245099)
Result = SUCCESS
bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1245099
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/security/TokenCache.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/security/TestTokenCache.java
> Change TokenCache's reading of the binary token file
> ----------------------------------------------------
>
> Key: MAPREDUCE-3849
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3849
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: security
> Affects Versions: 0.23.1, 0.24.0
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Fix For: 0.23.2
>
> Attachments: MAPREDUCE-3849-2.patch, MAPREDUCE-3849.patch
>
>
> When obtaining the tokens for a {{FileSystem}}, the {{TokenCache}} will read the binary token file if a token is not already in the {{Credentials}}. However, it will overwrite any existing tokens in the {{Credentials}} with the contents of the binary token file if a single token is missing. This may cause new tokens to be replaced with invalid/cancelled tokens from the binary file. The new tokens will not be canceled, and thus "leak" in the namenode until they expire.
> The binary tokens should be merged with, but not replace, existing tokens in the {{Credentials}}.
> The code that reads the binary token file is prefaced with:
> {code}
> //TODO: Need to come up with a better place to put
> //this block of code to do with reading the file
> {code}
> Also, the loading of the binary token file is the only reason that the {{TokenCache}} has to use {{getCanonicalService}}. If this linkage can be broken, then the 1-to-1 filesystem to token service coupling may be removed. And use of {{getCanonicalService}} can be removed in a subsequent jira.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3849) Change TokenCache's reading of
the binary token file
Posted by "Daryn Sharp (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daryn Sharp updated MAPREDUCE-3849:
-----------------------------------
Attachment: MAPREDUCE-3849-2.patch
Add more tests.
> Change TokenCache's reading of the binary token file
> ----------------------------------------------------
>
> Key: MAPREDUCE-3849
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3849
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: security
> Affects Versions: 0.23.1, 0.24.0
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Attachments: MAPREDUCE-3849-2.patch, MAPREDUCE-3849.patch
>
>
> When obtaining the tokens for a {{FileSystem}}, the {{TokenCache}} will read the binary token file if a token is not already in the {{Credentials}}. However, it will overwrite any existing tokens in the {{Credentials}} with the contents of the binary token file if a single token is missing. This may cause new tokens to be replaced with invalid/cancelled tokens from the binary file. The new tokens will not be canceled, and thus "leak" in the namenode until they expire.
> The binary tokens should be merged with, but not replace, existing tokens in the {{Credentials}}.
> The code that reads the binary token file is prefaced with:
> {code}
> //TODO: Need to come up with a better place to put
> //this block of code to do with reading the file
> {code}
> Also, the loading of the binary token file is the only reason that the {{TokenCache}} has to use {{getCanonicalService}}. If this linkage can be broken, then the 1-to-1 filesystem to token service coupling may be removed. And use of {{getCanonicalService}} can be removed in a subsequent jira.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3849) Change TokenCache's reading of
the binary token file
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209547#comment-13209547 ]
Hudson commented on MAPREDUCE-3849:
-----------------------------------
Integrated in Hadoop-Common-trunk-Commit #1735 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1735/])
MAPREDUCE-3849. Change TokenCache's reading of the binary token file (Daryn Sharp via bobby) (Revision 1245099)
Result = SUCCESS
bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1245099
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/security/TokenCache.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/security/TestTokenCache.java
> Change TokenCache's reading of the binary token file
> ----------------------------------------------------
>
> Key: MAPREDUCE-3849
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3849
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: security
> Affects Versions: 0.23.1, 0.24.0
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Fix For: 0.23.2
>
> Attachments: MAPREDUCE-3849-2.patch, MAPREDUCE-3849.patch
>
>
> When obtaining the tokens for a {{FileSystem}}, the {{TokenCache}} will read the binary token file if a token is not already in the {{Credentials}}. However, it will overwrite any existing tokens in the {{Credentials}} with the contents of the binary token file if a single token is missing. This may cause new tokens to be replaced with invalid/cancelled tokens from the binary file. The new tokens will not be canceled, and thus "leak" in the namenode until they expire.
> The binary tokens should be merged with, but not replace, existing tokens in the {{Credentials}}.
> The code that reads the binary token file is prefaced with:
> {code}
> //TODO: Need to come up with a better place to put
> //this block of code to do with reading the file
> {code}
> Also, the loading of the binary token file is the only reason that the {{TokenCache}} has to use {{getCanonicalService}}. If this linkage can be broken, then the 1-to-1 filesystem to token service coupling may be removed. And use of {{getCanonicalService}} can be removed in a subsequent jira.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3849) Change TokenCache's reading of
the binary token file
Posted by "Mahadev konar (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209562#comment-13209562 ]
Mahadev konar commented on MAPREDUCE-3849:
------------------------------------------
Daryn,
Did you test this with something like Oozie where the delegation tokens are loaded from a file in the mapper? If not we should test that out.
> Change TokenCache's reading of the binary token file
> ----------------------------------------------------
>
> Key: MAPREDUCE-3849
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3849
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: security
> Affects Versions: 0.23.1, 0.24.0
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Fix For: 0.23.2
>
> Attachments: MAPREDUCE-3849-2.patch, MAPREDUCE-3849.patch
>
>
> When obtaining the tokens for a {{FileSystem}}, the {{TokenCache}} will read the binary token file if a token is not already in the {{Credentials}}. However, it will overwrite any existing tokens in the {{Credentials}} with the contents of the binary token file if a single token is missing. This may cause new tokens to be replaced with invalid/cancelled tokens from the binary file. The new tokens will not be canceled, and thus "leak" in the namenode until they expire.
> The binary tokens should be merged with, but not replace, existing tokens in the {{Credentials}}.
> The code that reads the binary token file is prefaced with:
> {code}
> //TODO: Need to come up with a better place to put
> //this block of code to do with reading the file
> {code}
> Also, the loading of the binary token file is the only reason that the {{TokenCache}} has to use {{getCanonicalService}}. If this linkage can be broken, then the 1-to-1 filesystem to token service coupling may be removed. And use of {{getCanonicalService}} can be removed in a subsequent jira.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3849) Change TokenCache's reading of
the binary token file
Posted by "Daryn Sharp (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daryn Sharp updated MAPREDUCE-3849:
-----------------------------------
Target Version/s: 0.23.0, 0.24.0 (was: 0.24.0, 0.23.0)
Status: Patch Available (was: Open)
> Change TokenCache's reading of the binary token file
> ----------------------------------------------------
>
> Key: MAPREDUCE-3849
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3849
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: security
> Affects Versions: 0.23.1, 0.24.0
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Attachments: MAPREDUCE-3849.patch
>
>
> When obtaining the tokens for a {{FileSystem}}, the {{TokenCache}} will read the binary token file if a token is not already in the {{Credentials}}. However, it will overwrite any existing tokens in the {{Credentials}} with the contents of the binary token file if a single token is missing. This may cause new tokens to be replaced with invalid/cancelled tokens from the binary file. The new tokens will not be canceled, and thus "leak" in the namenode until they expire.
> The binary tokens should be merged with, but not replace, existing tokens in the {{Credentials}}.
> The code that reads the binary token file is prefaced with:
> {code}
> //TODO: Need to come up with a better place to put
> //this block of code to do with reading the file
> {code}
> Also, the loading of the binary token file is the only reason that the {{TokenCache}} has to use {{getCanonicalService}}. If this linkage can be broken, then the 1-to-1 filesystem to token service coupling may be removed. And use of {{getCanonicalService}} can be removed in a subsequent jira.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3849) Change TokenCache's reading of
the binary token file
Posted by "Robert Joseph Evans (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207027#comment-13207027 ]
Robert Joseph Evans commented on MAPREDUCE-3849:
------------------------------------------------
Also have you run any tests on clusters with this change? Have you tried to run jobs with security enabled?
> Change TokenCache's reading of the binary token file
> ----------------------------------------------------
>
> Key: MAPREDUCE-3849
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3849
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: security
> Affects Versions: 0.23.1, 0.24.0
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Attachments: MAPREDUCE-3849.patch
>
>
> When obtaining the tokens for a {{FileSystem}}, the {{TokenCache}} will read the binary token file if a token is not already in the {{Credentials}}. However, it will overwrite any existing tokens in the {{Credentials}} with the contents of the binary token file if a single token is missing. This may cause new tokens to be replaced with invalid/cancelled tokens from the binary file. The new tokens will not be canceled, and thus "leak" in the namenode until they expire.
> The binary tokens should be merged with, but not replace, existing tokens in the {{Credentials}}.
> The code that reads the binary token file is prefaced with:
> {code}
> //TODO: Need to come up with a better place to put
> //this block of code to do with reading the file
> {code}
> Also, the loading of the binary token file is the only reason that the {{TokenCache}} has to use {{getCanonicalService}}. If this linkage can be broken, then the 1-to-1 filesystem to token service coupling may be removed. And use of {{getCanonicalService}} can be removed in a subsequent jira.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3849) Change TokenCache's reading of
the binary token file
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13210267#comment-13210267 ]
Hudson commented on MAPREDUCE-3849:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk #993 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/993/])
MAPREDUCE-3849. Change TokenCache's reading of the binary token file (Daryn Sharp via bobby) (Revision 1245099)
Result = SUCCESS
bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1245099
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/security/TokenCache.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/security/TestTokenCache.java
> Change TokenCache's reading of the binary token file
> ----------------------------------------------------
>
> Key: MAPREDUCE-3849
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3849
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: security
> Affects Versions: 0.23.1, 0.24.0
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Fix For: 0.23.2
>
> Attachments: MAPREDUCE-3849-2.patch, MAPREDUCE-3849.patch
>
>
> When obtaining the tokens for a {{FileSystem}}, the {{TokenCache}} will read the binary token file if a token is not already in the {{Credentials}}. However, it will overwrite any existing tokens in the {{Credentials}} with the contents of the binary token file if a single token is missing. This may cause new tokens to be replaced with invalid/cancelled tokens from the binary file. The new tokens will not be canceled, and thus "leak" in the namenode until they expire.
> The binary tokens should be merged with, but not replace, existing tokens in the {{Credentials}}.
> The code that reads the binary token file is prefaced with:
> {code}
> //TODO: Need to come up with a better place to put
> //this block of code to do with reading the file
> {code}
> Also, the loading of the binary token file is the only reason that the {{TokenCache}} has to use {{getCanonicalService}}. If this linkage can be broken, then the 1-to-1 filesystem to token service coupling may be removed. And use of {{getCanonicalService}} can be removed in a subsequent jira.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3849) Change TokenCache's reading of
the binary token file
Posted by "Robert Joseph Evans (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209536#comment-13209536 ]
Robert Joseph Evans commented on MAPREDUCE-3849:
------------------------------------------------
Thanks Daryn the patch looks good +1
> Change TokenCache's reading of the binary token file
> ----------------------------------------------------
>
> Key: MAPREDUCE-3849
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3849
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: security
> Affects Versions: 0.23.1, 0.24.0
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Attachments: MAPREDUCE-3849-2.patch, MAPREDUCE-3849.patch
>
>
> When obtaining the tokens for a {{FileSystem}}, the {{TokenCache}} will read the binary token file if a token is not already in the {{Credentials}}. However, it will overwrite any existing tokens in the {{Credentials}} with the contents of the binary token file if a single token is missing. This may cause new tokens to be replaced with invalid/cancelled tokens from the binary file. The new tokens will not be canceled, and thus "leak" in the namenode until they expire.
> The binary tokens should be merged with, but not replace, existing tokens in the {{Credentials}}.
> The code that reads the binary token file is prefaced with:
> {code}
> //TODO: Need to come up with a better place to put
> //this block of code to do with reading the file
> {code}
> Also, the loading of the binary token file is the only reason that the {{TokenCache}} has to use {{getCanonicalService}}. If this linkage can be broken, then the 1-to-1 filesystem to token service coupling may be removed. And use of {{getCanonicalService}} can be removed in a subsequent jira.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3849) Change TokenCache's reading of
the binary token file
Posted by "Daryn Sharp (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209732#comment-13209732 ]
Daryn Sharp commented on MAPREDUCE-3849:
----------------------------------------
No, I did not test against oozie. I didn't think of it nor do I have convenient access, but I'll see what I can do. The binary token file is still read in, but do you see a problem in the patch? Or just general concern?
On a related note, I saw a recent patch that clears the binary token file setting from the job's conf during submission. That may cause problems for oozie but I don't (yet) understand the internals of oozie well enough.
> Change TokenCache's reading of the binary token file
> ----------------------------------------------------
>
> Key: MAPREDUCE-3849
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3849
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: security
> Affects Versions: 0.23.1, 0.24.0
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Fix For: 0.23.2
>
> Attachments: MAPREDUCE-3849-2.patch, MAPREDUCE-3849.patch
>
>
> When obtaining the tokens for a {{FileSystem}}, the {{TokenCache}} will read the binary token file if a token is not already in the {{Credentials}}. However, it will overwrite any existing tokens in the {{Credentials}} with the contents of the binary token file if a single token is missing. This may cause new tokens to be replaced with invalid/cancelled tokens from the binary file. The new tokens will not be canceled, and thus "leak" in the namenode until they expire.
> The binary tokens should be merged with, but not replace, existing tokens in the {{Credentials}}.
> The code that reads the binary token file is prefaced with:
> {code}
> //TODO: Need to come up with a better place to put
> //this block of code to do with reading the file
> {code}
> Also, the loading of the binary token file is the only reason that the {{TokenCache}} has to use {{getCanonicalService}}. If this linkage can be broken, then the 1-to-1 filesystem to token service coupling may be removed. And use of {{getCanonicalService}} can be removed in a subsequent jira.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3849) Change TokenCache's reading of
the binary token file
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13210214#comment-13210214 ]
Hudson commented on MAPREDUCE-3849:
-----------------------------------
Integrated in Hadoop-Hdfs-0.23-Build #171 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/171/])
svn merge -c 1245099 from trunk to 0.23 FIXES MAPREDUCE-3849 Change TokenCache's reading of the binary token file (Daryn Sharp via bobby) (Revision 1245102)
Result = SUCCESS
bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1245102
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/security/TokenCache.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/security/TestTokenCache.java
> Change TokenCache's reading of the binary token file
> ----------------------------------------------------
>
> Key: MAPREDUCE-3849
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3849
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: security
> Affects Versions: 0.23.1, 0.24.0
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Fix For: 0.23.2
>
> Attachments: MAPREDUCE-3849-2.patch, MAPREDUCE-3849.patch
>
>
> When obtaining the tokens for a {{FileSystem}}, the {{TokenCache}} will read the binary token file if a token is not already in the {{Credentials}}. However, it will overwrite any existing tokens in the {{Credentials}} with the contents of the binary token file if a single token is missing. This may cause new tokens to be replaced with invalid/cancelled tokens from the binary file. The new tokens will not be canceled, and thus "leak" in the namenode until they expire.
> The binary tokens should be merged with, but not replace, existing tokens in the {{Credentials}}.
> The code that reads the binary token file is prefaced with:
> {code}
> //TODO: Need to come up with a better place to put
> //this block of code to do with reading the file
> {code}
> Also, the loading of the binary token file is the only reason that the {{TokenCache}} has to use {{getCanonicalService}}. If this linkage can be broken, then the 1-to-1 filesystem to token service coupling may be removed. And use of {{getCanonicalService}} can be removed in a subsequent jira.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3849) Change TokenCache's reading of
the binary token file
Posted by "Daryn Sharp (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daryn Sharp updated MAPREDUCE-3849:
-----------------------------------
Target Version/s: 0.23.0, 0.24.0 (was: 0.24.0, 0.23.0)
Issue Type: Improvement (was: Bug)
> Change TokenCache's reading of the binary token file
> ----------------------------------------------------
>
> Key: MAPREDUCE-3849
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3849
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: security
> Affects Versions: 0.23.1, 0.24.0
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
>
> When obtaining the tokens for a {{FileSystem}}, the {{TokenCache}} will read the binary token file if a token is not already in the {{Credentials}}. However, it will overwrite any existing tokens in the {{Credentials}} with the contents of the binary token file if a single token is missing. This may cause new tokens to be replaced with invalid/cancelled tokens from the binary file. The new tokens will not be canceled, and thus "leak" in the namenode until they expire.
> The binary tokens should be merged with, but not replace, existing tokens in the {{Credentials}}.
> The code that reads the binary token file is prefaced with:
> {code}
> //TODO: Need to come up with a better place to put
> //this block of code to do with reading the file
> {code}
> Also, the loading of the binary token file is the only reason that the {{TokenCache}} has to use {{getCanonicalService}}. If this linkage can be broken, then the 1-to-1 filesystem to token service coupling may be removed. And use of {{getCanonicalService}} can be removed in a subsequent jira.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3849) Change TokenCache's reading of
the binary token file
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209554#comment-13209554 ]
Hudson commented on MAPREDUCE-3849:
-----------------------------------
Integrated in Hadoop-Hdfs-0.23-Commit #546 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/546/])
svn merge -c 1245099 from trunk to 0.23 FIXES MAPREDUCE-3849 Change TokenCache's reading of the binary token file (Daryn Sharp via bobby) (Revision 1245102)
Result = SUCCESS
bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1245102
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/security/TokenCache.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/security/TestTokenCache.java
> Change TokenCache's reading of the binary token file
> ----------------------------------------------------
>
> Key: MAPREDUCE-3849
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3849
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: security
> Affects Versions: 0.23.1, 0.24.0
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Fix For: 0.23.2
>
> Attachments: MAPREDUCE-3849-2.patch, MAPREDUCE-3849.patch
>
>
> When obtaining the tokens for a {{FileSystem}}, the {{TokenCache}} will read the binary token file if a token is not already in the {{Credentials}}. However, it will overwrite any existing tokens in the {{Credentials}} with the contents of the binary token file if a single token is missing. This may cause new tokens to be replaced with invalid/cancelled tokens from the binary file. The new tokens will not be canceled, and thus "leak" in the namenode until they expire.
> The binary tokens should be merged with, but not replace, existing tokens in the {{Credentials}}.
> The code that reads the binary token file is prefaced with:
> {code}
> //TODO: Need to come up with a better place to put
> //this block of code to do with reading the file
> {code}
> Also, the loading of the binary token file is the only reason that the {{TokenCache}} has to use {{getCanonicalService}}. If this linkage can be broken, then the 1-to-1 filesystem to token service coupling may be removed. And use of {{getCanonicalService}} can be removed in a subsequent jira.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3849) Change TokenCache's reading of
the binary token file
Posted by "Daryn Sharp (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207030#comment-13207030 ]
Daryn Sharp commented on MAPREDUCE-3849:
----------------------------------------
TestLinuxResourceCalculatorPlugin was fixed last week by MAPREDUCE-3852.
I ran MR tests, some of which appear to fake out security.
> Change TokenCache's reading of the binary token file
> ----------------------------------------------------
>
> Key: MAPREDUCE-3849
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3849
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: security
> Affects Versions: 0.23.1, 0.24.0
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Attachments: MAPREDUCE-3849.patch
>
>
> When obtaining the tokens for a {{FileSystem}}, the {{TokenCache}} will read the binary token file if a token is not already in the {{Credentials}}. However, it will overwrite any existing tokens in the {{Credentials}} with the contents of the binary token file if a single token is missing. This may cause new tokens to be replaced with invalid/cancelled tokens from the binary file. The new tokens will not be canceled, and thus "leak" in the namenode until they expire.
> The binary tokens should be merged with, but not replace, existing tokens in the {{Credentials}}.
> The code that reads the binary token file is prefaced with:
> {code}
> //TODO: Need to come up with a better place to put
> //this block of code to do with reading the file
> {code}
> Also, the loading of the binary token file is the only reason that the {{TokenCache}} has to use {{getCanonicalService}}. If this linkage can be broken, then the 1-to-1 filesystem to token service coupling may be removed. And use of {{getCanonicalService}} can be removed in a subsequent jira.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3849) Change TokenCache's reading of
the binary token file
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209589#comment-13209589 ]
Hudson commented on MAPREDUCE-3849:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #1747 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1747/])
MAPREDUCE-3849. Change TokenCache's reading of the binary token file (Daryn Sharp via bobby) (Revision 1245099)
Result = ABORTED
bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1245099
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/security/TokenCache.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/security/TestTokenCache.java
> Change TokenCache's reading of the binary token file
> ----------------------------------------------------
>
> Key: MAPREDUCE-3849
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3849
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: security
> Affects Versions: 0.23.1, 0.24.0
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Fix For: 0.23.2
>
> Attachments: MAPREDUCE-3849-2.patch, MAPREDUCE-3849.patch
>
>
> When obtaining the tokens for a {{FileSystem}}, the {{TokenCache}} will read the binary token file if a token is not already in the {{Credentials}}. However, it will overwrite any existing tokens in the {{Credentials}} with the contents of the binary token file if a single token is missing. This may cause new tokens to be replaced with invalid/cancelled tokens from the binary file. The new tokens will not be canceled, and thus "leak" in the namenode until they expire.
> The binary tokens should be merged with, but not replace, existing tokens in the {{Credentials}}.
> The code that reads the binary token file is prefaced with:
> {code}
> //TODO: Need to come up with a better place to put
> //this block of code to do with reading the file
> {code}
> Also, the loading of the binary token file is the only reason that the {{TokenCache}} has to use {{getCanonicalService}}. If this linkage can be broken, then the 1-to-1 filesystem to token service coupling may be removed. And use of {{getCanonicalService}} can be removed in a subsequent jira.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3849) Change TokenCache's reading of
the binary token file
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13208331#comment-13208331 ]
Hadoop QA commented on MAPREDUCE-3849:
--------------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12514555/MAPREDUCE-3849-2.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1855//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1855//console
This message is automatically generated.
> Change TokenCache's reading of the binary token file
> ----------------------------------------------------
>
> Key: MAPREDUCE-3849
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3849
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: security
> Affects Versions: 0.23.1, 0.24.0
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Attachments: MAPREDUCE-3849-2.patch, MAPREDUCE-3849.patch
>
>
> When obtaining the tokens for a {{FileSystem}}, the {{TokenCache}} will read the binary token file if a token is not already in the {{Credentials}}. However, it will overwrite any existing tokens in the {{Credentials}} with the contents of the binary token file if a single token is missing. This may cause new tokens to be replaced with invalid/cancelled tokens from the binary file. The new tokens will not be canceled, and thus "leak" in the namenode until they expire.
> The binary tokens should be merged with, but not replace, existing tokens in the {{Credentials}}.
> The code that reads the binary token file is prefaced with:
> {code}
> //TODO: Need to come up with a better place to put
> //this block of code to do with reading the file
> {code}
> Also, the loading of the binary token file is the only reason that the {{TokenCache}} has to use {{getCanonicalService}}. If this linkage can be broken, then the 1-to-1 filesystem to token service coupling may be removed. And use of {{getCanonicalService}} can be removed in a subsequent jira.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira