[jira] [Created] (HDFS-15289) Allow viewfs mounts with hdfs scheme and centralized mount table

["Uma Maheswara Rao G (Jira)" <ji...@apache.org>]

Uma Maheswara Rao G created HDFS-15289:
------------------------------------------

             Summary: Allow viewfs mounts with hdfs scheme and centralized mount table
                 Key: HDFS-15289
                 URL: https://issues.apache.org/jira/browse/HDFS-15289
             Project: Hadoop HDFS
          Issue Type: New Feature
          Components: fs
    Affects Versions: 3.2.0
            Reporter: Uma Maheswara Rao G
            Assignee: Uma Maheswara Rao G
             Fix For: 3.4.0


ViewFS provides flexibility to mount different filesystem types with mount points configuration table. Additionally viewFS provides flexibility to configure any fs (not only HDFS) scheme in mount table mapping. This approach is solving the scalability problems, but users need to reconfigure the filesystem to ViewFS and to its scheme.  This will be problematic in the case of paths persisted in meta stores, ex: Hive. In systems like Hive, it will store uris in meta store. So, changing the file system scheme will create a burden to upgrade/recreate meta stores. In our experience many users are not ready to change that.  

Router based federation is another implementation to provide coordinated mount points for HDFS federation clusters. Even though this provides flexibility to handle mount points easily, this will not allow other(non-HDFS) file systems to mount. So, this does not solve the purpose when users want to mount external(non-HDFS) filesystems.

So, the problem here is: Even though many users want to adapt to the scalable fs options available, technical challenges of changing schemes (ex: in meta stores) in deployments are obstructing them. 

So, we propose to allow hdfs scheme in ViewFS like client side mount system and provision user to create mount links without changing URI paths. 
I will upload detailed design doc shortly.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org