You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-dev@hadoop.apache.org by "Ed Kohlwey (JIRA)" <ji...@apache.org> on 2010/09/20 16:42:32 UTC
[jira] Created: (HDFS-1412) Broken symlinks break the local
filesystem implementation of HDFS on linux
Broken symlinks break the local filesystem implementation of HDFS on linux
--------------------------------------------------------------------------
Key: HDFS-1412
URL: https://issues.apache.org/jira/browse/HDFS-1412
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 0.20.2
Environment: Ubuntu 9.10 Lucid
Reporter: Ed Kohlwey
When calling listStatus() on a directory containing broken symlinks, a FileNotFound exception is thrown.
The problem is that even though File.list() returns an entry for the symlink, calling File.exists() on a file with that path will return false.
I would suggest either checking that files exist in RawLocalFileSystem.listStatus() before calling getFileStatus(), and if a file doesn't exist don't put in in the result (less work), or modify the RawLocalFileSystem implementation to treat broken symlinks as files of length 0 (more work), or add an exists() method to FileStatus (perhaps even more work, and involves changing the API).
Here's the relevant section of the stack trace.
java.io.FileNotFoundException: File XXX does not exist.
at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:361)
at org.apache.hadoop.fs.RawLocalFileSystem.listStatus(RawLocalFileSystem.java:290)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:721)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:746)
at org.apache.hadoop.fs.ChecksumFileSystem.listStatus(ChecksumFileSystem.java:465)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HDFS-1412) Broken symlinks break the local
filesystem implementation of HDFS on linux
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Todd Lipcon resolved HDFS-1412.
-------------------------------
Resolution: Duplicate
Dup of HADOOP-6378
> Broken symlinks break the local filesystem implementation of HDFS on linux
> --------------------------------------------------------------------------
>
> Key: HDFS-1412
> URL: https://issues.apache.org/jira/browse/HDFS-1412
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 0.20.2
> Environment: Ubuntu 9.10 Lucid
> Reporter: Ed Kohlwey
>
> When calling listStatus() on a directory containing broken symlinks, a FileNotFound exception is thrown.
> The problem is that even though File.list() returns an entry for the symlink, calling File.exists() on a file with that path will return false.
> I would suggest either checking that files exist in RawLocalFileSystem.listStatus() before calling getFileStatus(), and if a file doesn't exist don't put in in the result (less work), or modify the RawLocalFileSystem implementation to treat broken symlinks as files of length 0 (more work), or add an exists() method to FileStatus (perhaps even more work, and involves changing the API).
> Here's the relevant section of the stack trace.
> java.io.FileNotFoundException: File XXX does not exist.
> at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:361)
> at org.apache.hadoop.fs.RawLocalFileSystem.listStatus(RawLocalFileSystem.java:290)
> at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:721)
> at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:746)
> at org.apache.hadoop.fs.ChecksumFileSystem.listStatus(ChecksumFileSystem.java:465)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.