You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Steve Loughran (JIRA)" <ji...@apache.org> on 2008/06/18 17:38:45 UTC
[jira] Created: (HADOOP-3591)
org.apache.hadoop.fs.FileUtil.checkDependencies may want to use .equals()
instead of ==
org.apache.hadoop.fs.FileUtil.checkDependencies may want to use .equals() instead of ==
---------------------------------------------------------------------------------------
Key: HADOOP-3591
URL: https://issues.apache.org/jira/browse/HADOOP-3591
Project: Hadoop Core
Issue Type: Bug
Components: fs
Affects Versions: 0.18.0
Reporter: Steve Loughran
Priority: Minor
org.apache.hadoop.fs.FileUtil.checkDependencies checks for source overwriting dest by first checking for filesystem equality:
if (srcFS == dstFS) {
..more checks here
}
So, its assuming that you can check filesystem by pointer equality, whereas if you create two filesystems and initialise them, they will still refer to the same remoteFS.
Consider assuming URI equality ==filesystem equality, and either to do a FileSystem.equals() that checks that , or do the check in FileUtil.getDependencies
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3591)
org.apache.hadoop.fs.FileUtil.checkDependencies may want to use .equals()
instead of ==
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12608145#action_12608145 ]
Doug Cutting commented on HADOOP-3591:
--------------------------------------
Comparing URIs for equality is certainly better than comparing FileSystem instances, but I'm not convinced this is in general a solvalble problem: we won't ever be able to guarantee that two URIs don't refer to the same file. An FTP server could refer to the same file as an HTTP server, etc.
I would support switching things from FileSystem instance equality to URI equality as an improvement, but I don't think it is worth our while to invest in a more elaborate equality mechanism.
> org.apache.hadoop.fs.FileUtil.checkDependencies may want to use .equals() instead of ==
> ---------------------------------------------------------------------------------------
>
> Key: HADOOP-3591
> URL: https://issues.apache.org/jira/browse/HADOOP-3591
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.18.0
> Reporter: Steve Loughran
> Priority: Minor
>
> org.apache.hadoop.fs.FileUtil.checkDependencies checks for source overwriting dest by first checking for filesystem equality:
> if (srcFS == dstFS) {
> ..more checks here
> }
> So, its assuming that you can check filesystem by pointer equality, whereas if you create two filesystems and initialise them, they will still refer to the same remoteFS.
> Consider assuming URI equality ==filesystem equality, and either to do a FileSystem.equals() that checks that , or do the check in FileUtil.getDependencies
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3591)
org.apache.hadoop.fs.FileUtil.checkDependencies may want to use .equals()
instead of ==
Posted by "Steve Loughran (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12608034#action_12608034 ]
Steve Loughran commented on HADOOP-3591:
----------------------------------------
well, some of the FileUtil and FsShell operations do equality tests either on instance (very bad) or uri. If we need a robust way to distinguish filesystems *and URI is inadequate*, then the only alternative would be for the far end to provide some unique ID. This could be a persistent ID of the FS itself, or it could be something created whenenver a name node or other filesystem comes up.
> org.apache.hadoop.fs.FileUtil.checkDependencies may want to use .equals() instead of ==
> ---------------------------------------------------------------------------------------
>
> Key: HADOOP-3591
> URL: https://issues.apache.org/jira/browse/HADOOP-3591
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.18.0
> Reporter: Steve Loughran
> Priority: Minor
>
> org.apache.hadoop.fs.FileUtil.checkDependencies checks for source overwriting dest by first checking for filesystem equality:
> if (srcFS == dstFS) {
> ..more checks here
> }
> So, its assuming that you can check filesystem by pointer equality, whereas if you create two filesystems and initialise them, they will still refer to the same remoteFS.
> Consider assuming URI equality ==filesystem equality, and either to do a FileSystem.equals() that checks that , or do the check in FileUtil.getDependencies
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3591)
org.apache.hadoop.fs.FileUtil.checkDependencies may want to use .equals()
instead of ==
Posted by "Steve Loughran (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12607616#action_12607616 ]
Steve Loughran commented on HADOOP-3591:
----------------------------------------
Note that FsShell does compare filesystems by URI
> org.apache.hadoop.fs.FileUtil.checkDependencies may want to use .equals() instead of ==
> ---------------------------------------------------------------------------------------
>
> Key: HADOOP-3591
> URL: https://issues.apache.org/jira/browse/HADOOP-3591
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.18.0
> Reporter: Steve Loughran
> Priority: Minor
>
> org.apache.hadoop.fs.FileUtil.checkDependencies checks for source overwriting dest by first checking for filesystem equality:
> if (srcFS == dstFS) {
> ..more checks here
> }
> So, its assuming that you can check filesystem by pointer equality, whereas if you create two filesystems and initialise them, they will still refer to the same remoteFS.
> Consider assuming URI equality ==filesystem equality, and either to do a FileSystem.equals() that checks that , or do the check in FileUtil.getDependencies
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3591)
org.apache.hadoop.fs.FileUtil.checkDependencies may want to use .equals()
instead of ==
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12607760#action_12607760 ]
Doug Cutting commented on HADOOP-3591:
--------------------------------------
Depending on FileSystem equality is a bug. Even though FileSystem instances are cached, it is still possible to have different instances that refer to the same files if different user credentials are used to create them. Equality of FileSystem.getUri() should be more reliable, but is still not perfect, as foo://joe@bar.com/ and bar://bob@1.2.3.4/ could refer to the same files, since foo and bar could be different protocols to access the same data, and bar.com's address could be 1.2.3.4.
> org.apache.hadoop.fs.FileUtil.checkDependencies may want to use .equals() instead of ==
> ---------------------------------------------------------------------------------------
>
> Key: HADOOP-3591
> URL: https://issues.apache.org/jira/browse/HADOOP-3591
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.18.0
> Reporter: Steve Loughran
> Priority: Minor
>
> org.apache.hadoop.fs.FileUtil.checkDependencies checks for source overwriting dest by first checking for filesystem equality:
> if (srcFS == dstFS) {
> ..more checks here
> }
> So, its assuming that you can check filesystem by pointer equality, whereas if you create two filesystems and initialise them, they will still refer to the same remoteFS.
> Consider assuming URI equality ==filesystem equality, and either to do a FileSystem.equals() that checks that , or do the check in FileUtil.getDependencies
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.