You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2016/10/04 18:35:22 UTC
[jira] [Commented] (NIFI-2828) SelectHiveQL and PutHiveQL fail with
NoClassDefFoundError when using HTTP transport
[ https://issues.apache.org/jira/browse/NIFI-2828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15546250#comment-15546250 ]
ASF GitHub Bot commented on NIFI-2828:
--------------------------------------
Github user bbende commented on the issue:
https://github.com/apache/nifi/pull/1075
Tried this out on against an HDP 2.4 sandbox and got an exception that boiled down to:
```
Caused by: java.io.IOException: No FileSystem for scheme: hdfs
at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2584) ~[hadoop-common-2.6.2.jar:na]
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2591) ~[hadoop-common-2.6.2.jar:na]
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:91) ~[hadoop-common-2.6.2.jar:na]
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2630) ~[hadoop-common-2.6.2.jar:na]
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2612) ~[hadoop-common-2.6.2.jar:na]
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:370) ~[hadoop-common-2.6.2.jar:na]
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296) ~[hadoop-common-2.6.2.jar:na]
at org.apache.hadoop.hive.ql.io.orc.OrcRecordUpdater.<init>(OrcRecordUpdater.java:221) ~[hive-exec-1.2.1.jar:1.2.1]
at org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat.getRecordUpdater(OrcOutputFormat.java:292) ~[hive-exec-1.2.1.jar:1.2.1]
at org.apache.hive.hcatalog.streaming.AbstractRecordWriter.createRecordUpdater(AbstractRecordWriter.java:141) ~[hive-hcatalog-streaming-1.2.1.jar:1.2.1]
at org.apache.hive.hcatalog.streaming.AbstractRecordWriter.newBatch(AbstractRecordWriter.java:121) ~[hive-hcatalog-streaming-1.2.1.jar:1.2.1]
... 10 common frames omitted
```
Comparing the JARs that get included with Hadoop Libraries NAR vs the Hive NAR there appear to be some differences.
Hadoop Libraries NAR:
hadoop-annotations-2.6.2.jar
hadoop-auth-2.6.2.jar
hadoop-client-2.6.2.jar
hadoop-common-2.6.2.jar
hadoop-hdfs-2.6.2.jar
hadoop-mapreduce-client-app-2.6.2.jar
hadoop-mapreduce-client-common-2.6.2.jar
hadoop-mapreduce-client-core-2.6.2.jar
hadoop-mapreduce-client-jobclient-2.6.2.jar
hadoop-mapreduce-client-shuffle-2.6.2.jar
hadoop-yarn-api-2.6.2.jar
hadoop-yarn-client-2.6.2.jar
hadoop-yarn-common-2.6.2.jar
hadoop-yarn-server-common-2.6.2.jar
Hive NAR:
hadoop-annotations-2.6.2.jar
hadoop-auth-2.6.2.jar
hadoop-common-2.6.2.jar
hadoop-mapreduce-client-core-2.6.2.jar
hadoop-yarn-api-2.6.2.jar
hadoop-yarn-common-2.6.2.jar
hadoop-yarn-server-applicationhistoryservice-2.6.0.jar
hadoop-yarn-server-common-2.6.0.jar
hadoop-yarn-server-resourcemanager-2.6.0.jar
hadoop-yarn-server-web-proxy-2.6.0.jar
I think the Hive NAR at least needs the hadoop-hdfs jar, but not sure what else.
> SelectHiveQL and PutHiveQL fail with NoClassDefFoundError when using HTTP transport
> ------------------------------------------------------------------------------------
>
> Key: NIFI-2828
> URL: https://issues.apache.org/jira/browse/NIFI-2828
> Project: Apache NiFi
> Issue Type: Bug
> Reporter: Joey Frazee
> Assignee: Matt Burgess
> Fix For: 1.1.0
>
>
> SelectHiveQL and PutHiveQL don't currently work with HTTP transport. There appears to be a class loader problem resulting in java.lang.NoClassDefFoundError: Could not initialize class org.apache.http.conn.ssl.SSLConnectionSocketFactory.
> This looks like a conflict with the Apache commons httpclient version in hadoop-common. Removing the hadoop-libraries .nar dependency and provided scope for hadoop-common appears to fix the issue, but I haven't done any rigorous testing so I'm not sure if there are other consequences or not.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)