[jira] Updated: (HDFS-303) Make contracts of LocalFileSystem and DistributedFileSystem consistent

["Tom White (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HDFS-303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tom White updated HDFS-303:
---------------------------

    Attachment: HDFS-303-common-test-case.patch

Following HDFS-538, it would be good to remove these inconsistencies. Here is what I think we should do for each of them (the attached test shows how LocalFileSystem fails compared to HDFS). I'd be grateful to get others thoughts on this.

bq. 1. When trying to make a directory under an existing file, HDFS throws an IOException while LocalFileSystem doesn't.
An IOException seems good here.
bq. 2. The FileSytem#listStatus(Path) method returns null for a non-existent file on HDFS, while LocalFileSytem returns an empty FileStatus array.
Fixed in  HDFS-538
bq. 3. When trying to rename a non-existent path, LocalFileSystem throws an FileNotFoundException, while HDFS returns false.
FileNotFoundException seems appropriate here.
bq. 4. When renaming a file or directory to a non-existent directory (e.g. /a/b to /c/d, where /c doesn't exist) LocalFileSystem succeeds (returns true) while HDFS fails (false).
HDFS seems more correct here.
bq. 5. When renaming a file (or directory) as an existing file (or directory) LocalFileSystem succeeds (returns true) while HDFS fails (false).

There are two cases:
i. Rename file to existing file
The question here is do we allow file overwrites? LocalFileSystem and POSIX rename (http://www.opengroup.org/onlinepubs/009695399/functions/rename.html) both do, but HDFS historically has not. Is there a reason not to allow this?

ii. Rename directory to existing directory
If the existing directory is empty then it is reasonable for the rename to succeed. Otherwise it should fail. (This is consistent with POSIX)

Another problem is that rename() and mkdirs() may either return false or throw an IOException to indicate failure. Clients need to check both cases to be sure if the operation succeeded. Ideally we would just use one mechanism (exceptions), but that is not so easy to migrate to since we can't overload on return type (boolean and void).

Also, LocalFileSystem delegates rename() to java.io.File#renameTo(), which is platform-dependent. For this reason some of these corner cases should be explicitly handled by LocalFileSystem itself.

> Make contracts of LocalFileSystem and DistributedFileSystem consistent
> ----------------------------------------------------------------------
>
>                 Key: HDFS-303
>                 URL: https://issues.apache.org/jira/browse/HDFS-303
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Tom White
>         Attachments: hadoop-4114.patch, HDFS-303-common-test-case.patch
>
>
> There are a number of edge cases that the two file system implementations handle differently. In particular:
> * When trying to make a directory under an existing file, HDFS throws an IOException while LocalFileSystem doesn't.
> * The FileSytem#listStatus(Path) method returns null for a non-existent file on HDFS, while LocalFileSytem returns an empty FileStatus array.
> * When trying to rename a non-existent path, LocalFileSystem throws an IOException, while HDFS returns false.
> * When renaming a file or directory to a non-existent directory (e.g. /a/b to /c/d, where /c doesn't exist) LocalFileSystem succeeds (returns true) while HDFS fails (false).
> * When renaming a file (or directory) as an existing file (or directory) LocalFileSystem succeeds (returns true) while HDFS fails (false).
> We should document the expected behaviour for these cases in FileSystem's javadoc, and make sure all implementations conform to it.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.