You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Aihua Xu (JIRA)" <ji...@apache.org> on 2017/09/27 21:12:00 UTC
[jira] [Assigned] (HIVE-17624) MapredLocakTask running in separate
JVM could throw ClassNotFoundException
[ https://issues.apache.org/jira/browse/HIVE-17624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aihua Xu reassigned HIVE-17624:
-------------------------------
> MapredLocakTask running in separate JVM could throw ClassNotFoundException
> ---------------------------------------------------------------------------
>
> Key: HIVE-17624
> URL: https://issues.apache.org/jira/browse/HIVE-17624
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
> Affects Versions: 2.1.1
> Reporter: Aihua Xu
> Assignee: Aihua Xu
>
> {noformat}
> set hive.auto.convert.join=true;
> set hive.auto.convert.join.use.nonstaged=false;
> add jar hive-hcatalog-core.jar;
> drop table if exists t1;
> CREATE TABLE t1 (a string, b string)
> ROW FORMAT SERDE 'org.apache.hive.hcatalog.data.JsonSerDe';
> LOAD DATA LOCAL INPATH "data/files/sample.json" INTO TABLE t1;
> select * from t1 l join t1 r on l.a=r.a;
> {noformat}
> The join will use a MapJoin which uses MapredLocalTask in a separate JVM to load the table into a Hashmap. But hive doesn't pass added jar to the classpath in such JVM so the following exception is thrown.
> {noformat}
> org.apache.hadoop.hive.ql.metadata.HiveException: Failed with exception java.lang.ClassNotFoundException: org.apache.hive.hcatalog.data.JsonSerDejava.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.hive.hcatalog.data.JsonSerDe
> at org.apache.hadoop.hive.ql.plan.TableDesc.getDeserializerClass(TableDesc.java:72)
> at org.apache.hadoop.hive.ql.plan.TableDesc.getDeserializer(TableDesc.java:92)
> at org.apache.hadoop.hive.ql.exec.FetchOperator.setupOutputObjectInspector(FetchOperator.java:564)
> at org.apache.hadoop.hive.ql.exec.FetchOperator.initialize(FetchOperator.java:172)
> at org.apache.hadoop.hive.ql.exec.FetchOperator.<init>(FetchOperator.java:140)
> at org.apache.hadoop.hive.ql.exec.FetchOperator.<init>(FetchOperator.java:127)
> at org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask.initializeOperators(MapredLocalTask.java:462)
> at org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask.startForward(MapredLocalTask.java:390)
> at org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask.executeInProcess(MapredLocalTask.java:370)
> at org.apache.hadoop.hive.ql.exec.mr.ExecDriver.main(ExecDriver.java:756)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> Caused by: java.lang.ClassNotFoundException: org.apache.hive.hcatalog.data.JsonSerDe
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:270)
> at org.apache.hadoop.hive.ql.plan.TableDesc.getDeserializerClass(TableDesc.java:69)
> ... 15 more
> at org.apache.hadoop.hive.ql.exec.FetchOperator.setupOutputObjectInspector(FetchOperator.java:586)
> at org.apache.hadoop.hive.ql.exec.FetchOperator.initialize(FetchOperator.java:172)
> at org.apache.hadoop.hive.ql.exec.FetchOperator.<init>(FetchOperator.java:140)
> at org.apache.hadoop.hive.ql.exec.FetchOperator.<init>(FetchOperator.java:127)
> at org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask.initializeOperators(MapredLocalTask.java:462)
> at org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask.startForward(MapredLocalTask.java:390)
> at org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask.executeInProcess(MapredLocalTask.java:370)
> at org.apache.hadoop.hive.ql.exec.mr.ExecDriver.main(ExecDriver.java:756)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)