Unable to delete symlinks in HDFS via FileContext

[Dia Kharrat <dk...@gmail.com>]

Hi,

We have an HDFS setup (v2.0.0) managed by Cloudera. I'm having trouble
getting the FileStatus of a symlink or deleting it.

According to the documentation, FileContext#getFileLinkStatus() or
FileContext#delete() should operate on the symlink itself if the provided
path is a symlink.

However, what happens instead is, FileContext#getFileLinkStatus() or
delete() resolve the symlink and act on the target path. So, for example,
if "/tmp/symlink.file" points to "/tmp/target.file", doing this:

Path path = new Path("hdfs://nameservice1:8020/tmp/symlink.file");
fileContext.getFileLinkStatus(path).getPath();

returns a path of:

Path("hdfs://nameservice1:8020/tmp/target.file")

Similarly, fileContext.delete(path, true) deletes the target file
("/tmp/target.file") instead of the symlink itself. So, this behavior does
not match with the documentation.

What's interesting is, locally in pseudo-mode, the above works as expected.

Any ideas or pointers as to why FileContext#delete and
FileContext#getFileLinkStatus() are not operating correctly on the symlinks?

Thanks,
Dia

Re: Unable to delete symlinks in HDFS via FileContext

[Andrew Wang <an...@cloudera.com>]

Hi Dia,

That's definitely a weird one, let's try to figure out what's going on.

Can you possibly share the complete FileContext snippet you're using to do
this test? You could also try using fully-qualified URIs everywhere
(including for symlink creation), to remove any possible ambiguity.

Also, can you clarify the version of Hadoop you're using? CDH4.x? Apache
2.0.0? If you're using CDH, let's move this off hdfs-dev and over to
cdh-user@cloudera.org.

Best,
Andrew


On Wed, Apr 24, 2013 at 8:05 PM, Dia Kharrat <dk...@gmail.com> wrote:

> Hi,
>
> We have an HDFS setup (v2.0.0) managed by Cloudera. I'm having trouble
> getting the FileStatus of a symlink or deleting it.
>
> According to the documentation, FileContext#getFileLinkStatus() or
> FileContext#delete() should operate on the symlink itself if the provided
> path is a symlink.
>
> However, what happens instead is, FileContext#getFileLinkStatus() or
> delete() resolve the symlink and act on the target path. So, for example,
> if "/tmp/symlink.file" points to "/tmp/target.file", doing this:
>
> Path path = new Path("hdfs://nameservice1:8020/tmp/symlink.file");
> fileContext.getFileLinkStatus(path).getPath();
>
> returns a path of:
>
> Path("hdfs://nameservice1:8020/tmp/target.file")
>
> Similarly, fileContext.delete(path, true) deletes the target file
> ("/tmp/target.file") instead of the symlink itself. So, this behavior does
> not match with the documentation.
>
> What's interesting is, locally in pseudo-mode, the above works as expected.
>
> Any ideas or pointers as to why FileContext#delete and
> FileContext#getFileLinkStatus() are not operating correctly on the
> symlinks?
>
> Thanks,
> Dia
>