You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Mike Drob (JIRA)" <ji...@apache.org> on 2018/03/05 15:45:00 UTC
[jira] [Commented] (HBASE-20124) Make hbase-spark module work with
hadoop3
[ https://issues.apache.org/jira/browse/HBASE-20124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16386232#comment-16386232 ]
Mike Drob commented on HBASE-20124:
-----------------------------------
This is follow on work that we will do after HBASE-16179, right?
> Make hbase-spark module work with hadoop3
> -----------------------------------------
>
> Key: HBASE-20124
> URL: https://issues.apache.org/jira/browse/HBASE-20124
> Project: HBase
> Issue Type: Bug
> Reporter: Ted Yu
> Priority: Major
>
> The following error can be observed when running tests in hbase-spark module against hadoop3:
> {code}
> HBaseDStreamFunctionsSuite:
> *** RUN ABORTED ***
> java.lang.NoClassDefFoundError: org/apache/hadoop/ipc/ExternalCall
> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getStorageDirs(FSNamesystem.java:1464)
> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getNamespaceDirs(FSNamesystem.java:1444)
> at org.apache.hadoop.hdfs.MiniDFSCluster.createNameNodesAndSetConf(MiniDFSCluster.java:939)
> at org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:815)
> at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:746)
> at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniDFSCluster(HBaseTestingUtility.java:668)
> at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniDFSCluster(HBaseTestingUtility.java:640)
> at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:979)
> at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:859)
> at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:853)
> ...
> Cause: java.lang.ClassNotFoundException: org.apache.hadoop.ipc.ExternalCall
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:338)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getStorageDirs(FSNamesystem.java:1464)
> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getNamespaceDirs(FSNamesystem.java:1444)
> at org.apache.hadoop.hdfs.MiniDFSCluster.createNameNodesAndSetConf(MiniDFSCluster.java:939)
> at org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:815)
> at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:746)
> at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniDFSCluster(HBaseTestingUtility.java:668)
> {code}
> The dependency tree shows mixture of hadoop 2.7.4 and hadoop3 for the hbase-spark module.
> This should be addressed by adding proper profile in pom.xml
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)