You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Hao Ren <h....@claravista.fr> on 2013/08/07 11:50:11 UTC
Issue: join two tables containing Array.
Hi,
I encounter a problem when joining two tables which contain Array field.
Some test queries:
drop table t1;
create table t1 as
select 1 as id, Array('1','2') as list, 2 as nb from t_data limit 1;
drop table t2;
create table t2 as
select 1 as id, Array('3','4') as list, 2 as nb from t_data limit 1;
select * from t1 join t2 on(t1.id = t2.id)
The last join query ends with an exception:
java.lang.ClassCastException:
org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryArray cannot be cast
to [Ljava.lang.Object;
at
org.apache.hadoop.hive.serde2.objectinspector.StandardListObjectInspector.getList(StandardListObjectInspector.java:98)
at
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe.serialize(LazySimpleSerDe.java:434)
at
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe.serializeField(LazySimpleSerDe.java:381)
at
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe.serialize(LazySimpleSerDe.java:365)
at
org.apache.hadoop.hive.ql.exec.FileSinkOperator.processOp(FileSinkOperator.java:568)
at
shark.execution.FileSinkOperator$$anonfun$processPartition$1.apply(FileSinkOperator.scala:73)
at
shark.execution.FileSinkOperator$$anonfun$processPartition$1.apply(FileSinkOperator.scala:72)
at scala.collection.Iterator$class.foreach(Iterator.scala:772)
at scala.collection.Iterator$$anon$19.foreach(Iterator.scala:399)
at
shark.execution.FileSinkOperator.processPartition(FileSinkOperator.scala:72)
at
shark.execution.FileSinkOperator$.writeFiles$1(FileSinkOperator.scala:133)
at
shark.execution.FileSinkOperator$$anonfun$executeProcessFileSinkPartition$1.apply(FileSinkOperator.scala:138)
at
shark.execution.FileSinkOperator$$anonfun$executeProcessFileSinkPartition$1.apply(FileSinkOperator.scala:138)
at spark.scheduler.ResultTask.run(ResultTask.scala:77)
at spark.executor.Executor$TaskRunner.run(Executor.scala:98)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
Any workaround there ?
Hao
--
Hao Ren
ClaraVista
www.claravista.fr