You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Todd Lipcon (JIRA)" <ji...@apache.org> on 2009/11/17 00:13:40 UTC
[jira] Created: (HADOOP-6378) Broken symlinks in directory break
RawLocalFileSystem.listStatus
Broken symlinks in directory break RawLocalFileSystem.listStatus
----------------------------------------------------------------
Key: HADOOP-6378
URL: https://issues.apache.org/jira/browse/HADOOP-6378
Project: Hadoop Common
Issue Type: Bug
Components: fs
Affects Versions: 0.20.0, 0.21.0, 0.22.0
Reporter: Todd Lipcon
I have a broken unix symlink in my home directory. listStatus on that directory fails with a FileNotFoundException, which is unexpected.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6378) Broken symlinks in directory break
RawLocalFileSystem.listStatus
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12778636#action_12778636 ]
Todd Lipcon commented on HADOOP-6378:
-------------------------------------
I have a broken symlink in my home dir (an emacs lock file):
{noformat}
todd@todd-laptop:~$ ls -l .#caltrain.html
lrwxrwxrwx 1 todd todd 21 2009-04-08 19:51 .#caltrain.html -> todd@todd-laptop.6714
{noformat}
listStatus fails with the following exception (modified FsShell to actually show stack):
{noformat}
todd@todd-laptop:~/git/hadoop-common$ ./bin/hadoop fs -ls /home/todd/
java.io.FileNotFoundException: File file:/home/todd/.#caltrain.html does not exist.
at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:420)
at org.apache.hadoop.fs.RawLocalFileSystem.listStatus(RawLocalFileSystem.java:311)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1058)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1083)
at org.apache.hadoop.fs.ChecksumFileSystem.listStatus(ChecksumFileSystem.java:473)
at org.apache.hadoop.fs.FsShell.shellListStatus(FsShell.java:1251)
at org.apache.hadoop.fs.FsShell.ls(FsShell.java:601)
at org.apache.hadoop.fs.FsShell.ls(FsShell.java:590)
at org.apache.hadoop.fs.FsShell.doall(FsShell.java:1647)
at org.apache.hadoop.fs.FsShell.run(FsShell.java:1867)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.hadoop.fs.FsShell.main(FsShell.java:1973)
ls: could not get listing for 'file:/home/todd'
{noformat}
> Broken symlinks in directory break RawLocalFileSystem.listStatus
> ----------------------------------------------------------------
>
> Key: HADOOP-6378
> URL: https://issues.apache.org/jira/browse/HADOOP-6378
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.20.0, 0.21.0, 0.22.0
> Reporter: Todd Lipcon
>
> I have a broken unix symlink in my home directory. listStatus on that directory fails with a FileNotFoundException, which is unexpected.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6378) Broken symlinks in directory break
RawLocalFileSystem.listStatus
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12794735#action_12794735 ]
Todd Lipcon commented on HADOOP-6378:
-------------------------------------
This is slightly tricky to resolve. In listStatus, we can catch FileNotFoundException after calling getFileStatus. When that occurs, we have two options: (a) we just don't include that file in the listStatus results, or (b) we include that file with a FileStatus that indicates a 0 length file with 000 permissions. Unfortunately, Java <7 doesn't have any kind of lstat equivalent.
> Broken symlinks in directory break RawLocalFileSystem.listStatus
> ----------------------------------------------------------------
>
> Key: HADOOP-6378
> URL: https://issues.apache.org/jira/browse/HADOOP-6378
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.20.0, 0.21.0, 0.22.0
> Reporter: Todd Lipcon
>
> I have a broken unix symlink in my home directory. listStatus on that directory fails with a FileNotFoundException, which is unexpected.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6378) Broken symlinks in directory break
RawLocalFileSystem.listStatus
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12795500#action_12795500 ]
Eli Collins commented on HADOOP-6378:
-------------------------------------
I think we should do option (a) plus log a warning, I don't see why users would want us to fake up a FileStatus representing the link unless they could explicitly ask about the link itself (which they can't because FileContext doesn't have an lstat equivalent).
fwiw in the symlink patch getFileStatus on a dangling link yields a FNF exception (to maintain compatibility) while _getFileLinkStatus_ yields a FileStatus representing the dangling link (see testDanglingLink in TestLocalFsLink.java for the particular FileStatus field values).
> Broken symlinks in directory break RawLocalFileSystem.listStatus
> ----------------------------------------------------------------
>
> Key: HADOOP-6378
> URL: https://issues.apache.org/jira/browse/HADOOP-6378
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.20.0, 0.21.0, 0.22.0
> Reporter: Todd Lipcon
>
> I have a broken unix symlink in my home directory. listStatus on that directory fails with a FileNotFoundException, which is unexpected.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.