You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Prashanth Menon <pr...@gmail.com> on 2013/02/25 20:25:03 UTC
Can't join external tables with different serde's

Hi all,

I have two external tables stored in HDFS one using a JSON SerDe
(specifically this one https://github.com/rcongiu/Hive-JSON-Serde) and
another table in regular comma-separated form.  When I try to perform an
inner join on the two tables, I consistenly get this error:

java.lang.RuntimeException:
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row { ... row data ... }
    at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:161)
    at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:405)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:336)
    at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1126)
    at org.apache.hadoop.mapred.Child.main(Child.java:264)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
Error while processing row { ... row data ... }
    at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:548)
    at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:143)
    ... 8 more
Caused by: java.lang.ClassCastException:
org.apache.hadoop.hive.serde2.lazy.LazyStruct cannot be cast to
[Ljava.lang.Object;
    at
org.apache.hadoop.hive.serde2.objectinspector.UnionStructObjectInspector.getStructFieldsDataAsList(UnionStructObjectInspector.java:141)
    at
org.apache.hadoop.hive.ql.exec.LateralViewJoinOperator.processOp(LateralViewJoinOperator.java:128)
    at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
    at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
    at
org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:70)
    at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
    at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
    at
org.apache.hadoop.hive.ql.exec.LateralViewForwardOperator.processOp(LateralViewForwardOperator.java:37)
    at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
    at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
    at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:83)
    at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
    at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
    at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:529)
    ... 9 more
2013-02-25 14:15:52,818 INFO org.apache.hadoop.mapred.Task: Runnning
cleanup for the task

Any ideas?