[jira] [Created] (HBASE-22951) [HBCK2] hbase hbck throws IOE "No FileSystem for scheme: hdfs"

["stack (Jira)" <ji...@apache.org>]

stack created HBASE-22951:
-----------------------------

             Summary: [HBCK2] hbase hbck throws IOE "No FileSystem for scheme: hdfs"
                 Key: HBASE-22951
                 URL: https://issues.apache.org/jira/browse/HBASE-22951
             Project: HBase
          Issue Type: Bug
            Reporter: stack


Input appreciated on this one.

If I do the below, passing a config that is pointing at a HDFS, I get the below (If I run w/o, hbck just picks up the wrong fs -- the local fs).

{code}
$ /vagrant/hbase/bin/hbase --config hbase-conf  hbck
2019-08-30 05:04:54,467 WARN  [main] util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Exception in thread "main" java.io.IOException: No FileSystem for scheme: hdfs
        at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2799)
        at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2810)
        at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:100)
        at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2849)
        at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2831)
        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:389)
        at org.apache.hadoop.fs.Path.getFileSystem(Path.java:356)
        at org.apache.hadoop.hbase.util.CommonFSUtils.getRootDir(CommonFSUtils.java:361)
        at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:3605)
{code}

Its because the CLASSPATH is carefully curated so as to use shaded client only; there are no hdfs classes on the CLASSPATH intentionally.

So, how to fix? Happens whether hbck1 or hbck2 (you have to do a hdfs operation for hbck2 to trigger same issue).

Could be careful in hbck2 and note that if fs operation, you need to add hdfs jars to CLASSPATH so hbck2 can go against hdfs.

If add the ' --internal-classpath' flag, then all classes are put on the CLASSPATH for hbck(2) (including the hdfs client jar which got the hdfs implementation after 2.7.2 was released) and stuff 'works'.

Could edit the bin/hbase script and make it so hdfs classes are added to the hbck CLASSPATH? Could see if could do hdfs client-only?

Anyways, putting this up for now. Others may have opinions. Thanks.





--
This message was sent by Atlassian Jira
(v8.3.2#803003)