You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Devaraj Das (JIRA)" <ji...@apache.org> on 2010/02/08 09:15:32 UTC
[jira] Created: (HADOOP-6545) Cached FileSystem objects can lead to
wrong token being used in setting up connections
Cached FileSystem objects can lead to wrong token being used in setting up connections
--------------------------------------------------------------------------------------
Key: HADOOP-6545
URL: https://issues.apache.org/jira/browse/HADOOP-6545
Project: Hadoop Common
Issue Type: Bug
Components: security
Affects Versions: 0.22.0
Reporter: Devaraj Das
Fix For: 0.22.0
The FileSystem class caches the filesystem objects that it creates for users. For some cases, e.g., if the filesystem object is actually a DistributedFileSystem, it also has an associated RPC client and hence an UGI for the respective user. This could lead to issues to do with using the right credentials when connecting with the namenode. The credentials in the UGI is never updated (even if the user in question now has new credentials) and in case the cached UGI's credentials have expired, this would lead to authentication error whenever there is a re-authentication (in the process of re-establishing connection to the namenode).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6545) Cached FileSystem objects can lead
to wrong token being used in setting up connections
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12833682#action_12833682 ]
Hadoop QA commented on HADOOP-6545:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12435830/6545-1.patch
against trunk revision 909806.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/5/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/5/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/5/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/5/console
This message is automatically generated.
> Cached FileSystem objects can lead to wrong token being used in setting up connections
> --------------------------------------------------------------------------------------
>
> Key: HADOOP-6545
> URL: https://issues.apache.org/jira/browse/HADOOP-6545
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
> Affects Versions: 0.22.0
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.22.0
>
> Attachments: 6545-1.patch
>
>
> The FileSystem class caches the filesystem objects that it creates for users. For some cases, e.g., if the filesystem object is actually a DistributedFileSystem, it also has an associated RPC client and hence an UGI for the respective user. This could lead to issues to do with using the right credentials when connecting with the namenode. The credentials in the UGI is never updated (even if the user in question now has new credentials) and in case the cached UGI's credentials have expired, this would lead to authentication error whenever there is a re-authentication (in the process of re-establishing connection to the namenode).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6545) Cached FileSystem objects can lead to
wrong token being used in setting up connections
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated HADOOP-6545:
--------------------------------
Attachment: 6545-bp20.patch
The backport for Y20. Not for commit here.
> Cached FileSystem objects can lead to wrong token being used in setting up connections
> --------------------------------------------------------------------------------------
>
> Key: HADOOP-6545
> URL: https://issues.apache.org/jira/browse/HADOOP-6545
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
> Affects Versions: 0.22.0
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.22.0
>
> Attachments: 6545-1.patch, 6545-2.patch, 6545-bp20.patch
>
>
> The FileSystem class caches the filesystem objects that it creates for users. For some cases, e.g., if the filesystem object is actually a DistributedFileSystem, it also has an associated RPC client and hence an UGI for the respective user. This could lead to issues to do with using the right credentials when connecting with the namenode. The credentials in the UGI is never updated (even if the user in question now has new credentials) and in case the cached UGI's credentials have expired, this would lead to authentication error whenever there is a re-authentication (in the process of re-establishing connection to the namenode).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6545) Cached FileSystem objects can lead to
wrong token being used in setting up connections
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated HADOOP-6545:
--------------------------------
Attachment: 6545-1.patch
Looks like the correct solution is to have the UGI as part of the key in the FileSystem cache. Attaching a patch that has this change.
> Cached FileSystem objects can lead to wrong token being used in setting up connections
> --------------------------------------------------------------------------------------
>
> Key: HADOOP-6545
> URL: https://issues.apache.org/jira/browse/HADOOP-6545
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
> Affects Versions: 0.22.0
> Reporter: Devaraj Das
> Fix For: 0.22.0
>
> Attachments: 6545-1.patch
>
>
> The FileSystem class caches the filesystem objects that it creates for users. For some cases, e.g., if the filesystem object is actually a DistributedFileSystem, it also has an associated RPC client and hence an UGI for the respective user. This could lead to issues to do with using the right credentials when connecting with the namenode. The credentials in the UGI is never updated (even if the user in question now has new credentials) and in case the cached UGI's credentials have expired, this would lead to authentication error whenever there is a re-authentication (in the process of re-establishing connection to the namenode).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6545) Cached FileSystem objects can lead to
wrong token being used in setting up connections
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated HADOOP-6545:
--------------------------------
Status: Open (was: Patch Available)
> Cached FileSystem objects can lead to wrong token being used in setting up connections
> --------------------------------------------------------------------------------------
>
> Key: HADOOP-6545
> URL: https://issues.apache.org/jira/browse/HADOOP-6545
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
> Affects Versions: 0.22.0
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.22.0
>
> Attachments: 6545-1.patch
>
>
> The FileSystem class caches the filesystem objects that it creates for users. For some cases, e.g., if the filesystem object is actually a DistributedFileSystem, it also has an associated RPC client and hence an UGI for the respective user. This could lead to issues to do with using the right credentials when connecting with the namenode. The credentials in the UGI is never updated (even if the user in question now has new credentials) and in case the cached UGI's credentials have expired, this would lead to authentication error whenever there is a re-authentication (in the process of re-establishing connection to the namenode).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6545) Cached FileSystem objects can lead
to wrong token being used in setting up connections
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12836358#action_12836358 ]
Hudson commented on HADOOP-6545:
--------------------------------
Integrated in Hadoop-Common-trunk #256 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk/256/])
. Changes the Key for the FileSystem cache to be UGI. Contributed by Devaraj Das.
> Cached FileSystem objects can lead to wrong token being used in setting up connections
> --------------------------------------------------------------------------------------
>
> Key: HADOOP-6545
> URL: https://issues.apache.org/jira/browse/HADOOP-6545
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
> Affects Versions: 0.22.0
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.22.0
>
> Attachments: 6545-1.patch, 6545-2.patch, 6545-bp20.patch
>
>
> The FileSystem class caches the filesystem objects that it creates for users. For some cases, e.g., if the filesystem object is actually a DistributedFileSystem, it also has an associated RPC client and hence an UGI for the respective user. This could lead to issues to do with using the right credentials when connecting with the namenode. The credentials in the UGI is never updated (even if the user in question now has new credentials) and in case the cached UGI's credentials have expired, this would lead to authentication error whenever there is a re-authentication (in the process of re-establishing connection to the namenode).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6545) Cached FileSystem objects can lead to
wrong token being used in setting up connections
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated HADOOP-6545:
--------------------------------
Attachment: 6545-2.patch
This patch adds tests.
> Cached FileSystem objects can lead to wrong token being used in setting up connections
> --------------------------------------------------------------------------------------
>
> Key: HADOOP-6545
> URL: https://issues.apache.org/jira/browse/HADOOP-6545
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
> Affects Versions: 0.22.0
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.22.0
>
> Attachments: 6545-1.patch, 6545-2.patch
>
>
> The FileSystem class caches the filesystem objects that it creates for users. For some cases, e.g., if the filesystem object is actually a DistributedFileSystem, it also has an associated RPC client and hence an UGI for the respective user. This could lead to issues to do with using the right credentials when connecting with the namenode. The credentials in the UGI is never updated (even if the user in question now has new credentials) and in case the cached UGI's credentials have expired, this would lead to authentication error whenever there is a re-authentication (in the process of re-establishing connection to the namenode).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6545) Cached FileSystem objects can lead to
wrong token being used in setting up connections
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated HADOOP-6545:
--------------------------------
Assignee: Devaraj Das
Status: Patch Available (was: Open)
Oh I changed the toString in FileSystem.Cache.Key to have UGI.toString within braces. This is because UGI's toString could be "effective-user via real-user", and i wanted to associate the two together via the braces..
> Cached FileSystem objects can lead to wrong token being used in setting up connections
> --------------------------------------------------------------------------------------
>
> Key: HADOOP-6545
> URL: https://issues.apache.org/jira/browse/HADOOP-6545
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
> Affects Versions: 0.22.0
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.22.0
>
> Attachments: 6545-1.patch
>
>
> The FileSystem class caches the filesystem objects that it creates for users. For some cases, e.g., if the filesystem object is actually a DistributedFileSystem, it also has an associated RPC client and hence an UGI for the respective user. This could lead to issues to do with using the right credentials when connecting with the namenode. The credentials in the UGI is never updated (even if the user in question now has new credentials) and in case the cached UGI's credentials have expired, this would lead to authentication error whenever there is a re-authentication (in the process of re-establishing connection to the namenode).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6545) Cached FileSystem objects can lead to
wrong token being used in setting up connections
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated HADOOP-6545:
--------------------------------
Status: Patch Available (was: Open)
> Cached FileSystem objects can lead to wrong token being used in setting up connections
> --------------------------------------------------------------------------------------
>
> Key: HADOOP-6545
> URL: https://issues.apache.org/jira/browse/HADOOP-6545
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
> Affects Versions: 0.22.0
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.22.0
>
> Attachments: 6545-1.patch, 6545-2.patch
>
>
> The FileSystem class caches the filesystem objects that it creates for users. For some cases, e.g., if the filesystem object is actually a DistributedFileSystem, it also has an associated RPC client and hence an UGI for the respective user. This could lead to issues to do with using the right credentials when connecting with the namenode. The credentials in the UGI is never updated (even if the user in question now has new credentials) and in case the cached UGI's credentials have expired, this would lead to authentication error whenever there is a re-authentication (in the process of re-establishing connection to the namenode).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6545) Cached FileSystem objects can lead
to wrong token being used in setting up connections
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12836257#action_12836257 ]
Hudson commented on HADOOP-6545:
--------------------------------
Integrated in Hadoop-Common-trunk-Commit #179 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk-Commit/179/])
. Changes the Key for the FileSystem cache to be UGI. Contributed by Devaraj Das.
> Cached FileSystem objects can lead to wrong token being used in setting up connections
> --------------------------------------------------------------------------------------
>
> Key: HADOOP-6545
> URL: https://issues.apache.org/jira/browse/HADOOP-6545
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
> Affects Versions: 0.22.0
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.22.0
>
> Attachments: 6545-1.patch, 6545-2.patch, 6545-bp20.patch
>
>
> The FileSystem class caches the filesystem objects that it creates for users. For some cases, e.g., if the filesystem object is actually a DistributedFileSystem, it also has an associated RPC client and hence an UGI for the respective user. This could lead to issues to do with using the right credentials when connecting with the namenode. The credentials in the UGI is never updated (even if the user in question now has new credentials) and in case the cached UGI's credentials have expired, this would lead to authentication error whenever there is a re-authentication (in the process of re-establishing connection to the namenode).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6545) Cached FileSystem objects can lead
to wrong token being used in setting up connections
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12835655#action_12835655 ]
Owen O'Malley commented on HADOOP-6545:
---------------------------------------
+1
> Cached FileSystem objects can lead to wrong token being used in setting up connections
> --------------------------------------------------------------------------------------
>
> Key: HADOOP-6545
> URL: https://issues.apache.org/jira/browse/HADOOP-6545
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
> Affects Versions: 0.22.0
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.22.0
>
> Attachments: 6545-1.patch, 6545-2.patch
>
>
> The FileSystem class caches the filesystem objects that it creates for users. For some cases, e.g., if the filesystem object is actually a DistributedFileSystem, it also has an associated RPC client and hence an UGI for the respective user. This could lead to issues to do with using the right credentials when connecting with the namenode. The credentials in the UGI is never updated (even if the user in question now has new credentials) and in case the cached UGI's credentials have expired, this would lead to authentication error whenever there is a re-authentication (in the process of re-establishing connection to the namenode).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6545) Cached FileSystem objects can lead
to wrong token being used in setting up connections
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12833921#action_12833921 ]
Hadoop QA commented on HADOOP-6545:
-----------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12435891/6545-2.patch
against trunk revision 910169.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/361/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/361/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/361/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/361/console
This message is automatically generated.
> Cached FileSystem objects can lead to wrong token being used in setting up connections
> --------------------------------------------------------------------------------------
>
> Key: HADOOP-6545
> URL: https://issues.apache.org/jira/browse/HADOOP-6545
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
> Affects Versions: 0.22.0
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.22.0
>
> Attachments: 6545-1.patch, 6545-2.patch
>
>
> The FileSystem class caches the filesystem objects that it creates for users. For some cases, e.g., if the filesystem object is actually a DistributedFileSystem, it also has an associated RPC client and hence an UGI for the respective user. This could lead to issues to do with using the right credentials when connecting with the namenode. The credentials in the UGI is never updated (even if the user in question now has new credentials) and in case the cached UGI's credentials have expired, this would lead to authentication error whenever there is a re-authentication (in the process of re-establishing connection to the namenode).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6545) Cached FileSystem objects can lead to
wrong token being used in setting up connections
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated HADOOP-6545:
--------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
I just committed this.
> Cached FileSystem objects can lead to wrong token being used in setting up connections
> --------------------------------------------------------------------------------------
>
> Key: HADOOP-6545
> URL: https://issues.apache.org/jira/browse/HADOOP-6545
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
> Affects Versions: 0.22.0
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.22.0
>
> Attachments: 6545-1.patch, 6545-2.patch
>
>
> The FileSystem class caches the filesystem objects that it creates for users. For some cases, e.g., if the filesystem object is actually a DistributedFileSystem, it also has an associated RPC client and hence an UGI for the respective user. This could lead to issues to do with using the right credentials when connecting with the namenode. The credentials in the UGI is never updated (even if the user in question now has new credentials) and in case the cached UGI's credentials have expired, this would lead to authentication error whenever there is a re-authentication (in the process of re-establishing connection to the namenode).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.