You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Junfan Zhang (Jira)" <ji...@apache.org> on 2020/10/15 11:48:00 UTC
[jira] [Comment Edited] (HDFS-15289) Allow viewfs mounts with
HDFS/HCFS scheme and centralized mount table
[ https://issues.apache.org/jira/browse/HDFS-15289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17214628#comment-17214628 ]
Junfan Zhang edited comment on HDFS-15289 at 10/15/20, 11:47 AM:
-----------------------------------------------------------------
Hi [~umamaheswararao] Thanks your post. We have also implemented a similar file system and have already applied it online to solve the problems of hybrid-cloud architecture and cluster data migration . What I want to know is: Does {{ViewFSOverloadScheme}} support the mounting of different cluster paths?
was (Author: zuston):
Hi [~umamaheswararao] Thanks your post. We have also implemented a similar file system and have already applied it online to solve the problems of multi-cloud architecture and cluster migration . What I want to know is: Does {{ViewFSOverloadScheme}} support the mounting of different cluster paths?
> Allow viewfs mounts with HDFS/HCFS scheme and centralized mount table
> ---------------------------------------------------------------------
>
> Key: HDFS-15289
> URL: https://issues.apache.org/jira/browse/HDFS-15289
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: fs
> Affects Versions: 3.2.0
> Reporter: Uma Maheswara Rao G
> Assignee: Uma Maheswara Rao G
> Priority: Major
> Attachments: ViewFSOverloadScheme - V1.0.pdf, ViewFSOverloadScheme.png
>
>
> ViewFS provides flexibility to mount different filesystem types with mount points configuration table. This approach is solving the scalability problems, but users need to reconfigure the filesystem to ViewFS and to its scheme. This will be problematic in the case of paths persisted in meta stores, ex: Hive. In systems like Hive, it will store uris in meta store. So, changing the file system scheme will create a burden to upgrade/recreate meta stores. In our experience many users are not ready to change that.
> Router based federation is another implementation to provide coordinated mount points for HDFS federation clusters. Even though this provides flexibility to handle mount points easily, this will not allow other(non-HDFS) file systems to mount. So, this does not solve the purpose when users want to mount external(non-HDFS) filesystems.
> So, the problem here is: Even though many users want to adapt to the scalable fs options available, technical challenges of changing schemes (ex: in meta stores) in deployments are obstructing them.
> So, we propose to allow hdfs scheme in ViewFS like client side mount system and provision user to create mount links without changing URI paths.
> I will upload detailed design doc shortly.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org