[jira] [Created] (HDFS-15533) Provide DFS API compatible call, but use ViewFileSystemOverloadScheme inside

["Uma Maheswara Rao G (Jira)" <ji...@apache.org>]

Uma Maheswara Rao G created HDFS-15533:
------------------------------------------

             Summary: Provide DFS API compatible call, but use ViewFileSystemOverloadScheme inside
                 Key: HDFS-15533
                 URL: https://issues.apache.org/jira/browse/HDFS-15533
             Project: Hadoop HDFS
          Issue Type: Sub-task
          Components: dfs, viewfs
    Affects Versions: 3.4.0
            Reporter: Uma Maheswara Rao G
            Assignee: Uma Maheswara Rao G


I have been working on a thought from last week is that, we wanted to provide DFS compatible APIs with mount functionality. So, that existing DFS applications can work with out class cast issues.

When we tested with other components like Hive and HBase, I noticed some classcast issues.


{code:java}
HBase example:
java.lang.ClassCastException: org.apache.hadoop.fs.viewfs.ViewFileSystemOverloadScheme cannot be cast to org.apache.hadoop.hdfs.DistributedFileSystemjava.lang.ClassCastException: org.apache.hadoop.fs.viewfs.ViewFileSystemOverloadScheme cannot be cast to org.apache.hadoop.hdfs.DistributedFileSystem at org.apache.hadoop.hbase.util.FSUtils.getDFSHedgedReadMetrics(FSUtils.java:1748) at org.apache.hadoop.hbase.regionserver.MetricsRegionServerWrapperImpl.<init>(MetricsRegionServerWrapperImpl.java:146) at org.apache.hadoop.hbase.regionserver.HRegionServer.handleReportForDutyResponse(HRegionServer.java:1594) at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:1001) at java.lang.Thread.run(Thread.java:748){code}
{code:java}
Hive:
|io.AcidUtils|: Failed to get files with ID; using regular API: Only supported for DFS; got class org.apache.hadoop.fs.viewfs.ViewFileSystemOverloadScheme{code}
SO, here the implementation details are like follows:

We extended DistributedFileSystem and created a class called " ViewDistributedFileSystem"
This vfs=ViewFirstibutedFileSystem, try to initialize ViewFileSystemOverloadScheme. If success call will delegate to  vfs. If fails to initialize due to no mount points, or other errors, it will just fallback to regular DFS init. If users does not configure any mount, system will behave exactly like today's DFS. If there are mount points, vfs functionality will come under DFS.

I will a patch and will post it in some time.

 

 

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org