You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org> on 2008/03/12 02:48:46 UTC
[jira] Created: (HADOOP-3003) FileSystem cache key should be
updated after a FileSystem object is created
FileSystem cache key should be updated after a FileSystem object is created
---------------------------------------------------------------------------
Key: HADOOP-3003
URL: https://issues.apache.org/jira/browse/HADOOP-3003
Project: Hadoop Core
Issue Type: Bug
Components: fs
Reporter: Tsz Wo (Nicholas), SZE
Assignee: Tsz Wo (Nicholas), SZE
Fix For: 0.16.2
In FileSystem.get(uri, conf), it first creates a cache key from the uri and the conf and then lookups the corresponding FileSystem object in the cache. If the object is not found, it initializes a FileSystem object and put it to the cache with the key. However, during FileSystem creation, the conf might be modified. In such case, the key should be updated before putting it to the cache.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3003) FileSystem cache key should be
updated after a FileSystem object is created
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12577940#action_12577940 ]
Tsz Wo (Nicholas), SZE commented on HADOOP-3003:
------------------------------------------------
> ... updated user information. Is that right?
That's is true.
In HADOOP-2915, there is a discussion about using WeakReference in the cache, so that un-referenced fs could be closed automatically. In that case, the cache values have to be FsRef (extending WeakReference). You are right that we don't need FsRef currently. Do you want me to remove it or keep it for future use?
See https://issues.apache.org/jira/browse/HADOOP-2915?focusedCommentId=12575175#action_12575175
> FileSystem cache key should be updated after a FileSystem object is created
> ---------------------------------------------------------------------------
>
> Key: HADOOP-3003
> URL: https://issues.apache.org/jira/browse/HADOOP-3003
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Fix For: 0.16.2
>
> Attachments: 3003_20080311.patch
>
>
> In FileSystem.get(uri, conf), it first creates a cache key from the uri and the conf and then lookups the corresponding FileSystem object in the cache. If the object is not found, it initializes a FileSystem object and put it to the cache with the key. However, during FileSystem creation, the conf might be modified. In such case, the key should be updated before putting it to the cache.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3003) FileSystem cache key should be
updated after a FileSystem object is created
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HADOOP-3003:
-------------------------------------------
Fix Version/s: (was: 0.16.2)
0.17.0
Affects Version/s: 0.17.0
> FileSystem cache key should be updated after a FileSystem object is created
> ---------------------------------------------------------------------------
>
> Key: HADOOP-3003
> URL: https://issues.apache.org/jira/browse/HADOOP-3003
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.17.0
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Fix For: 0.17.0
>
> Attachments: 3003_20080311.patch
>
>
> In FileSystem.get(uri, conf), it first creates a cache key from the uri and the conf and then lookups the corresponding FileSystem object in the cache. If the object is not found, it initializes a FileSystem object and put it to the cache with the key. However, during FileSystem creation, the conf might be modified. In such case, the key should be updated before putting it to the cache.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3003) FileSystem cache key should be
updated after a FileSystem object is created
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12578229#action_12578229 ]
Hadoop QA commented on HADOOP-3003:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12377663/3003_20080311.patch
against trunk revision 619744.
@author +1. The patch does not contain any @author tags.
tests included +1. The patch appears to include 6 new or modified tests.
javadoc +1. The javadoc tool did not generate any warning messages.
javac +1. The applied patch does not generate any new javac compiler warnings.
release audit -1. The applied patch generated 192 release audit warnings (more than the trunk's current 0 warnings).
findbugs +1. The patch does not introduce any new Findbugs warnings.
core tests -1. The patch failed core unit tests.
contrib tests +1. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1952/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1952/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1952/artifact/trunk/build/test/checkstyle-errors.html
Release audit warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1952/artifact/trunk/current/releaseAuditDiffWarnings.txt
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1952/console
This message is automatically generated.
> FileSystem cache key should be updated after a FileSystem object is created
> ---------------------------------------------------------------------------
>
> Key: HADOOP-3003
> URL: https://issues.apache.org/jira/browse/HADOOP-3003
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.17.0
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Fix For: 0.17.0
>
> Attachments: 3003_20080311.patch
>
>
> In FileSystem.get(uri, conf), it first creates a cache key from the uri and the conf and then lookups the corresponding FileSystem object in the cache. If the object is not found, it initializes a FileSystem object and put it to the cache with the key. However, during FileSystem creation, the conf might be modified. In such case, the key should be updated before putting it to the cache.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3003) FileSystem cache key should be
updated after a FileSystem object is created
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12578018#action_12578018 ]
Tsz Wo (Nicholas), SZE commented on HADOOP-3003:
------------------------------------------------
It was lucky that this problem does not exist in 0.16.1 since the username in conf remains constant during FileSystem creation.
> FileSystem cache key should be updated after a FileSystem object is created
> ---------------------------------------------------------------------------
>
> Key: HADOOP-3003
> URL: https://issues.apache.org/jira/browse/HADOOP-3003
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.17.0
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Fix For: 0.17.0
>
> Attachments: 3003_20080311.patch
>
>
> In FileSystem.get(uri, conf), it first creates a cache key from the uri and the conf and then lookups the corresponding FileSystem object in the cache. If the object is not found, it initializes a FileSystem object and put it to the cache with the key. However, during FileSystem creation, the conf might be modified. In such case, the key should be updated before putting it to the cache.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3003) FileSystem cache key should be
updated after a FileSystem object is created
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12580753#action_12580753 ]
Hudson commented on HADOOP-3003:
--------------------------------
Integrated in Hadoop-trunk #434 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/434/])
> FileSystem cache key should be updated after a FileSystem object is created
> ---------------------------------------------------------------------------
>
> Key: HADOOP-3003
> URL: https://issues.apache.org/jira/browse/HADOOP-3003
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.16.2, 0.17.0
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Fix For: 0.16.2, 0.17.0
>
> Attachments: 3003_20080311.patch, 3003_20080313.patch, 3003_20080318.patch
>
>
> In FileSystem.get(uri, conf), it first creates a cache key from the uri and the conf and then lookups the corresponding FileSystem object in the cache. If the object is not found, it initializes a FileSystem object and put it to the cache with the key. However, during FileSystem creation, the conf might be modified. In such case, the key should be updated before putting it to the cache.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3003) FileSystem cache key should be
updated after a FileSystem object is created
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12580117#action_12580117 ]
Doug Cutting commented on HADOOP-3003:
--------------------------------------
+1
A thought: the filesystem cache keys are now <user,scheme,authority>. Since URIs have a user field, it is tempting to use a URI for the key, instead of defining a new class: scheme://user@host:port/. The only problem would be if the FileSystem URI's authority includes a user, especially if it is a different user. For example, the S3 FileSystem puts a user in the authority, but that should be consistent with the S3 login. Unfortunately, its probably not consistent with the way we're determining the username used by the cache, using UnixUserGroupInformation.login(). The bug is that login() should be per-filesystem, not static.
> FileSystem cache key should be updated after a FileSystem object is created
> ---------------------------------------------------------------------------
>
> Key: HADOOP-3003
> URL: https://issues.apache.org/jira/browse/HADOOP-3003
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.16.2, 0.17.0
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Fix For: 0.16.2, 0.17.0
>
> Attachments: 3003_20080311.patch, 3003_20080313.patch, 3003_20080318.patch
>
>
> In FileSystem.get(uri, conf), it first creates a cache key from the uri and the conf and then lookups the corresponding FileSystem object in the cache. If the object is not found, it initializes a FileSystem object and put it to the cache with the key. However, during FileSystem creation, the conf might be modified. In such case, the key should be updated before putting it to the cache.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3003) FileSystem cache key should be
updated after a FileSystem object is created
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HADOOP-3003:
-------------------------------------------
Status: Open (was: Patch Available)
> FileSystem cache key should be updated after a FileSystem object is created
> ---------------------------------------------------------------------------
>
> Key: HADOOP-3003
> URL: https://issues.apache.org/jira/browse/HADOOP-3003
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.17.0
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Fix For: 0.17.0
>
> Attachments: 3003_20080311.patch, 3003_20080313.patch
>
>
> In FileSystem.get(uri, conf), it first creates a cache key from the uri and the conf and then lookups the corresponding FileSystem object in the cache. If the object is not found, it initializes a FileSystem object and put it to the cache with the key. However, during FileSystem creation, the conf might be modified. In such case, the key should be updated before putting it to the cache.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3003) FileSystem cache key should be
updated after a FileSystem object is created
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12580363#action_12580363 ]
Hadoop QA commented on HADOOP-3003:
-----------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12378180/3003_20080318.patch
against trunk revision 619744.
@author +1. The patch does not contain any @author tags.
tests included +1. The patch appears to include 6 new or modified tests.
javadoc +1. The javadoc tool did not generate any warning messages.
javac +1. The applied patch does not generate any new javac compiler warnings.
release audit +1. The applied patch does not generate any new release audit warnings.
findbugs +1. The patch does not introduce any new Findbugs warnings.
core tests +1. The patch passed core unit tests.
contrib tests +1. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1995/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1995/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1995/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1995/console
This message is automatically generated.
> FileSystem cache key should be updated after a FileSystem object is created
> ---------------------------------------------------------------------------
>
> Key: HADOOP-3003
> URL: https://issues.apache.org/jira/browse/HADOOP-3003
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.16.2, 0.17.0
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Fix For: 0.16.2, 0.17.0
>
> Attachments: 3003_20080311.patch, 3003_20080313.patch, 3003_20080318.patch
>
>
> In FileSystem.get(uri, conf), it first creates a cache key from the uri and the conf and then lookups the corresponding FileSystem object in the cache. If the object is not found, it initializes a FileSystem object and put it to the cache with the key. However, during FileSystem creation, the conf might be modified. In such case, the key should be updated before putting it to the cache.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3003) FileSystem cache key should be
updated after a FileSystem object is created
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HADOOP-3003:
-------------------------------------------
Fix Version/s: 0.16.2
Affects Version/s: 0.16.2
Status: Patch Available (was: Open)
> FileSystem cache key should be updated after a FileSystem object is created
> ---------------------------------------------------------------------------
>
> Key: HADOOP-3003
> URL: https://issues.apache.org/jira/browse/HADOOP-3003
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.16.2, 0.17.0
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Fix For: 0.16.2, 0.17.0
>
> Attachments: 3003_20080311.patch, 3003_20080313.patch
>
>
> In FileSystem.get(uri, conf), it first creates a cache key from the uri and the conf and then lookups the corresponding FileSystem object in the cache. If the object is not found, it initializes a FileSystem object and put it to the cache with the key. However, during FileSystem creation, the conf might be modified. In such case, the key should be updated before putting it to the cache.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3003) FileSystem cache key should be
updated after a FileSystem object is created
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
dhruba borthakur updated HADOOP-3003:
-------------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
I just committed this. Thanks Nicholas!
> FileSystem cache key should be updated after a FileSystem object is created
> ---------------------------------------------------------------------------
>
> Key: HADOOP-3003
> URL: https://issues.apache.org/jira/browse/HADOOP-3003
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.16.2, 0.17.0
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Fix For: 0.16.2, 0.17.0
>
> Attachments: 3003_20080311.patch, 3003_20080313.patch, 3003_20080318.patch
>
>
> In FileSystem.get(uri, conf), it first creates a cache key from the uri and the conf and then lookups the corresponding FileSystem object in the cache. If the object is not found, it initializes a FileSystem object and put it to the cache with the key. However, during FileSystem creation, the conf might be modified. In such case, the key should be updated before putting it to the cache.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3003) FileSystem cache key should be
updated after a FileSystem object is created
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HADOOP-3003:
-------------------------------------------
Attachment: 3003_20080311.patch
3003_20080311.patch: should fix this problem.
> FileSystem cache key should be updated after a FileSystem object is created
> ---------------------------------------------------------------------------
>
> Key: HADOOP-3003
> URL: https://issues.apache.org/jira/browse/HADOOP-3003
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Fix For: 0.16.2
>
> Attachments: 3003_20080311.patch
>
>
> In FileSystem.get(uri, conf), it first creates a cache key from the uri and the conf and then lookups the corresponding FileSystem object in the cache. If the object is not found, it initializes a FileSystem object and put it to the cache with the key. However, during FileSystem creation, the conf might be modified. In such case, the key should be updated before putting it to the cache.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3003) FileSystem cache key should be
updated after a FileSystem object is created
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HADOOP-3003:
-------------------------------------------
Status: Open (was: Patch Available)
> FileSystem cache key should be updated after a FileSystem object is created
> ---------------------------------------------------------------------------
>
> Key: HADOOP-3003
> URL: https://issues.apache.org/jira/browse/HADOOP-3003
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.16.2, 0.17.0
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Fix For: 0.16.2, 0.17.0
>
> Attachments: 3003_20080311.patch, 3003_20080313.patch, 3003_20080318.patch
>
>
> In FileSystem.get(uri, conf), it first creates a cache key from the uri and the conf and then lookups the corresponding FileSystem object in the cache. If the object is not found, it initializes a FileSystem object and put it to the cache with the key. However, during FileSystem creation, the conf might be modified. In such case, the key should be updated before putting it to the cache.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3003) FileSystem cache key should be
updated after a FileSystem object is created
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HADOOP-3003:
-------------------------------------------
Attachment: 3003_20080313.patch
3003_20080313.patch: removed FsRef
> FileSystem cache key should be updated after a FileSystem object is created
> ---------------------------------------------------------------------------
>
> Key: HADOOP-3003
> URL: https://issues.apache.org/jira/browse/HADOOP-3003
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.17.0
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Fix For: 0.17.0
>
> Attachments: 3003_20080311.patch, 3003_20080313.patch
>
>
> In FileSystem.get(uri, conf), it first creates a cache key from the uri and the conf and then lookups the corresponding FileSystem object in the cache. If the object is not found, it initializes a FileSystem object and put it to the cache with the key. However, during FileSystem creation, the conf might be modified. In such case, the key should be updated before putting it to the cache.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3003) FileSystem cache key should be
updated after a FileSystem object is created
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12578020#action_12578020 ]
Doug Cutting commented on HADOOP-3003:
--------------------------------------
> Do you want me to remove it or keep it for future use?
Please remove it. We should not keep dead code around for future use, since plans often change and then we end up with just dead code.
> FileSystem cache key should be updated after a FileSystem object is created
> ---------------------------------------------------------------------------
>
> Key: HADOOP-3003
> URL: https://issues.apache.org/jira/browse/HADOOP-3003
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.17.0
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Fix For: 0.17.0
>
> Attachments: 3003_20080311.patch
>
>
> In FileSystem.get(uri, conf), it first creates a cache key from the uri and the conf and then lookups the corresponding FileSystem object in the cache. If the object is not found, it initializes a FileSystem object and put it to the cache with the key. However, during FileSystem creation, the conf might be modified. In such case, the key should be updated before putting it to the cache.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3003) FileSystem cache key should be
updated after a FileSystem object is created
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HADOOP-3003:
-------------------------------------------
Status: Patch Available (was: Open)
> FileSystem cache key should be updated after a FileSystem object is created
> ---------------------------------------------------------------------------
>
> Key: HADOOP-3003
> URL: https://issues.apache.org/jira/browse/HADOOP-3003
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.16.2, 0.17.0
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Fix For: 0.16.2, 0.17.0
>
> Attachments: 3003_20080311.patch, 3003_20080313.patch, 3003_20080318.patch
>
>
> In FileSystem.get(uri, conf), it first creates a cache key from the uri and the conf and then lookups the corresponding FileSystem object in the cache. If the object is not found, it initializes a FileSystem object and put it to the cache with the key. However, during FileSystem creation, the conf might be modified. In such case, the key should be updated before putting it to the cache.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3003) FileSystem cache key should be
updated after a FileSystem object is created
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HADOOP-3003:
-------------------------------------------
Attachment: 3003_20080318.patch
3003_20080318.patch: fixed some bugs.
> FileSystem cache key should be updated after a FileSystem object is created
> ---------------------------------------------------------------------------
>
> Key: HADOOP-3003
> URL: https://issues.apache.org/jira/browse/HADOOP-3003
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.16.2, 0.17.0
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Fix For: 0.16.2, 0.17.0
>
> Attachments: 3003_20080311.patch, 3003_20080313.patch, 3003_20080318.patch
>
>
> In FileSystem.get(uri, conf), it first creates a cache key from the uri and the conf and then lookups the corresponding FileSystem object in the cache. If the object is not found, it initializes a FileSystem object and put it to the cache with the key. However, during FileSystem creation, the conf might be modified. In such case, the key should be updated before putting it to the cache.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3003) FileSystem cache key should be
updated after a FileSystem object is created
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12579869#action_12579869 ]
Hadoop QA commented on HADOOP-3003:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12377835/3003_20080313.patch
against trunk revision 619744.
@author +1. The patch does not contain any @author tags.
tests included +1. The patch appears to include 6 new or modified tests.
javadoc +1. The javadoc tool did not generate any warning messages.
javac +1. The applied patch does not generate any new javac compiler warnings.
release audit +1. The applied patch does not generate any new release audit warnings.
findbugs +1. The patch does not introduce any new Findbugs warnings.
core tests -1. The patch failed core unit tests.
contrib tests -1. The patch failed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1985/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1985/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1985/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1985/console
This message is automatically generated.
> FileSystem cache key should be updated after a FileSystem object is created
> ---------------------------------------------------------------------------
>
> Key: HADOOP-3003
> URL: https://issues.apache.org/jira/browse/HADOOP-3003
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.16.2, 0.17.0
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Fix For: 0.16.2, 0.17.0
>
> Attachments: 3003_20080311.patch, 3003_20080313.patch
>
>
> In FileSystem.get(uri, conf), it first creates a cache key from the uri and the conf and then lookups the corresponding FileSystem object in the cache. If the object is not found, it initializes a FileSystem object and put it to the cache with the key. However, during FileSystem creation, the conf might be modified. In such case, the key should be updated before putting it to the cache.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3003) FileSystem cache key should be
updated after a FileSystem object is created
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12577902#action_12577902 ]
Doug Cutting commented on HADOOP-3003:
--------------------------------------
In particular, I think you mean that the user information in the configuration could have been modified by a login process, and that we want the updated user information. Is that right?
Also, I don't see why FsRef needs a Key, why cache values cannot just be FileSystem instances. The only place where the FsRef's key is used is in closeAll(), where map.entrySet() could be instead used to iterate through all key/value pairs, no?
> FileSystem cache key should be updated after a FileSystem object is created
> ---------------------------------------------------------------------------
>
> Key: HADOOP-3003
> URL: https://issues.apache.org/jira/browse/HADOOP-3003
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Fix For: 0.16.2
>
> Attachments: 3003_20080311.patch
>
>
> In FileSystem.get(uri, conf), it first creates a cache key from the uri and the conf and then lookups the corresponding FileSystem object in the cache. If the object is not found, it initializes a FileSystem object and put it to the cache with the key. However, during FileSystem creation, the conf might be modified. In such case, the key should be updated before putting it to the cache.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3003) FileSystem cache key should be
updated after a FileSystem object is created
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HADOOP-3003:
-------------------------------------------
Status: Patch Available (was: Open)
> FileSystem cache key should be updated after a FileSystem object is created
> ---------------------------------------------------------------------------
>
> Key: HADOOP-3003
> URL: https://issues.apache.org/jira/browse/HADOOP-3003
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Fix For: 0.16.2
>
> Attachments: 3003_20080311.patch
>
>
> In FileSystem.get(uri, conf), it first creates a cache key from the uri and the conf and then lookups the corresponding FileSystem object in the cache. If the object is not found, it initializes a FileSystem object and put it to the cache with the key. However, during FileSystem creation, the conf might be modified. In such case, the key should be updated before putting it to the cache.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.