[jira] [Commented] (HADOOP-13055) Implement linkMergeSlash for ViewFileSystem

["Andrew Wang (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-13055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15733822#comment-15733822 ] 

Andrew Wang commented on HADOOP-13055:
--------------------------------------

Thanks for working on this Zhe and Manoj. I gave the patch a look, had some high-level comments to go over first.

It looks like this is mostly config changes to allow mounting a filesystem on /. It doesn't enable nested mounts though, so if you mount an FS on /, that's all you get. This doesn't seem that useful, since the point of ViewFS is to combine multiple FS namespaces.

I interpret "linkMergeSlash" to mean having an FS mounted on /, but then allow the VFS mount table to override that. Essentially, nested mounts, special cased to mounting over the "/" mount.

Nested mounts are problematic though because we don't do path resolution fully on the client-side. This surfaces for recursive delete; once we've deferred it to a mounted filesystem, it won't pop back to VFS to get redirected to recurse through a nested mount. It won't behave like "rm -rf". I haven't looked at the full FS interface to see what else is out there.

My question is why "/" is called out as a special case for nested mounts. It doesn't seem easier to implement, and the semantic issues with recursive ops are still there. It's also different from a merge mount, in that merging merges multiple FSs, while IIUC this is mounting an FS *over* another FS.

[~sanjay.radia] could you provide any more historical perspective on the intent here?

HADOOP-8299 also provides a little hint from [~eli] in the description, where he refers to the idea of a "default NN" for a mount table. This lets us ignore the semantic differences with unix mounts, and simplifies the implementation as well (first try to resolve with the VFS mount table, then fallback to the default FS).

> Implement linkMergeSlash for ViewFileSystem
> -------------------------------------------
>
>                 Key: HADOOP-13055
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13055
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: fs, viewfs
>            Reporter: Zhe Zhang
>            Assignee: Manoj Govindassamy
>         Attachments: HADOOP-13055.00.patch, HADOOP-13055.01.patch, HADOOP-13055.02.patch, HADOOP-13055.03.patch, HADOOP-13055.04.patch
>
>
> In a multi-cluster environment it is sometimes useful to operate on the root / slash directory of an HDFS cluster. E.g., list all top level directories. Quoting the comment in {{ViewFs}}:
> {code}
>  *   A special case of the merge mount is where mount table's root is merged
>  *   with the root (slash) of another file system:
>  *   <ul>
>  *   <li>    fs.viewfs.mounttable.default.linkMergeSlash=hdfs://nn99/
>  *   </ul>
>  *   In this cases the root of the mount table is merged with the root of
>  *            <b>hdfs://nn99/ </b> 
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org