You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Hudson (JIRA)" <ji...@apache.org> on 2013/01/09 11:29:47 UTC
[jira] [Commented] (HIVE-2958) GROUP BY causing ClassCastException
[LazyDioInteger cannot be cast LazyInteger]
[ https://issues.apache.org/jira/browse/HIVE-2958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13548171#comment-13548171 ]
Hudson commented on HIVE-2958:
------------------------------
Integrated in Hive-trunk-hadoop2 #54 (See [https://builds.apache.org/job/Hive-trunk-hadoop2/54/])
HIVE-2958 [jira] GROUP BY causing ClassCastException [LazyDioInteger cannot be
cast LazyInteger]
(Navis Ryu via Ashutosh Chauhan)
Summary:
DPAL-1111 GROUP BY causing ClassCastException [LazyDioInteger cannot be cast
LazyInteger]
This relates to https://issues.apache.org/jira/browse/HIVE-1634.
The following work fine:
CREATE EXTERNAL TABLE tim_hbase_occurrence (
id int,
scientific_name string,
data_resource_id int
) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH
SERDEPROPERTIES (
"hbase.columns.mapping" = ":key#b,v:scientific_name#s,v:data_resource_id#b"
) TBLPROPERTIES(
"hbase.table.name" = "mini_occurrences",
"hbase.table.default.storage.type" = "binary"
);
SELECT * FROM tim_hbase_occurrence LIMIT 3;
SELECT * FROM tim_hbase_occurrence WHERE data_resource_id=1081 LIMIT 3;
However, the following fails:
SELECT data_resource_id, count(*) FROM tim_hbase_occurrence GROUP BY
data_resource_id;
The error given:
0 TS
2012-04-17 16:58:45,693 INFO org.apache.hadoop.hive.ql.exec.MapOperator:
Initialization Done 7 MAP
2012-04-17 16:58:45,714 INFO org.apache.hadoop.hive.ql.exec.MapOperator:
Processing alias tim_hbase_occurrence for file
hdfs://c1n2.gbif.org/user/hive/warehouse/tim_hbase_occurrence
2012-04-17 16:58:45,714 INFO org.apache.hadoop.hive.ql.exec.MapOperator: 7
forwarding 1 rows
2012-04-17 16:58:45,714 INFO org.apache.hadoop.hive.ql.exec.TableScanOperator: 0
forwarding 1 rows
2012-04-17 16:58:45,716 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: 1
forwarding 1 rows
2012-04-17 16:58:45,723 FATAL ExecMapper:
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row {"id":1444,"scientific_name":null,"data_resource_id":1081}
at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:548)
at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:143)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:391)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325)
at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157)
at org.apache.hadoop.mapred.Child.main(Child.java:264)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
java.lang.ClassCastException:
org.apache.hadoop.hive.serde2.lazydio.LazyDioInteger cannot be cast to
org.apache.hadoop.hive.serde2.lazy.LazyInteger
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.processOp(GroupByOperator.java:737)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
at
org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:83)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:529)
... 9 more
Caused by: java.lang.ClassCastException:
org.apache.hadoop.hive.serde2.lazydio.LazyDioInteger cannot be cast to
org.apache.hadoop.hive.serde2.lazy.LazyInteger
at
org.apache.hadoop.hive.serde2.lazy.objectinspector.primitive.LazyIntObjectInspector.copyObject(LazyIntObjectInspector.java:43)
at
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.copyToStandardObject(ObjectInspectorUtils.java:239)
at
org.apache.hadoop.hive.ql.exec.KeyWrapperFactory$ListKeyWrapper.deepCopyElements(KeyWrapperFactory.java:150)
at
org.apache.hadoop.hive.ql.exec.KeyWrapperFactory$ListKeyWrapper.deepCopyElements(KeyWrapperFactory.java:142)
at
org.apache.hadoop.hive.ql.exec.KeyWrapperFactory$ListKeyWrapper.copyKey(KeyWrapperFactory.java:119)
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.processHashAggr(GroupByOperator.java:750)
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.processOp(GroupByOperator.java:722)
... 18 more
Test Plan: EMPTY
Reviewers: JIRA, ashutoshc
Reviewed By: ashutoshc
Differential Revision: https://reviews.facebook.net/D2871 (Revision 1328157)
Result = ABORTED
hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1328157
Files :
* /hive/trunk/hbase-handler/src/test/queries/hbase_binary_external_table_queries.q
* /hive/trunk/hbase-handler/src/test/results/hbase_binary_external_table_queries.q.out
* /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazydio/LazyDioBoolean.java
* /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazydio/LazyDioByte.java
* /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazydio/LazyDioDouble.java
* /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazydio/LazyDioFloat.java
* /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazydio/LazyDioInteger.java
* /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazydio/LazyDioLong.java
* /hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazydio/LazyDioShort.java
> GROUP BY causing ClassCastException [LazyDioInteger cannot be cast LazyInteger]
> -------------------------------------------------------------------------------
>
> Key: HIVE-2958
> URL: https://issues.apache.org/jira/browse/HIVE-2958
> Project: Hive
> Issue Type: Bug
> Components: HBase Handler
> Affects Versions: 0.9.0
> Environment: HBase 0.90.4, Hive 0.90 snapshot (trunk) built today
> Reporter: Tim Robertson
> Assignee: Navis
> Priority: Blocker
> Fix For: 0.9.0
>
> Attachments: ASF.LICENSE.NOT.GRANTED--HIVE-2958.D2871.1.patch
>
>
> This relates to https://issues.apache.org/jira/browse/HIVE-1634.
> The following work fine:
> {code}
> CREATE EXTERNAL TABLE tim_hbase_occurrence (
> id int,
> scientific_name string,
> data_resource_id int
> ) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES (
> "hbase.columns.mapping" = ":key#b,v:scientific_name#s,v:data_resource_id#b"
> ) TBLPROPERTIES(
> "hbase.table.name" = "mini_occurrences",
> "hbase.table.default.storage.type" = "binary"
> );
> SELECT * FROM tim_hbase_occurrence LIMIT 3;
> SELECT * FROM tim_hbase_occurrence WHERE data_resource_id=1081 LIMIT 3;
> {code}
> However, the following fails:
> {code}
> SELECT data_resource_id, count(*) FROM tim_hbase_occurrence GROUP BY data_resource_id;
> {code}
> The error given:
> {code}
> 0 TS
> 2012-04-17 16:58:45,693 INFO org.apache.hadoop.hive.ql.exec.MapOperator: Initialization Done 7 MAP
> 2012-04-17 16:58:45,714 INFO org.apache.hadoop.hive.ql.exec.MapOperator: Processing alias tim_hbase_occurrence for file hdfs://c1n2.gbif.org/user/hive/warehouse/tim_hbase_occurrence
> 2012-04-17 16:58:45,714 INFO org.apache.hadoop.hive.ql.exec.MapOperator: 7 forwarding 1 rows
> 2012-04-17 16:58:45,714 INFO org.apache.hadoop.hive.ql.exec.TableScanOperator: 0 forwarding 1 rows
> 2012-04-17 16:58:45,716 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: 1 forwarding 1 rows
> 2012-04-17 16:58:45,723 FATAL ExecMapper: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {"id":1444,"scientific_name":null,"data_resource_id":1081}
> at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:548)
> at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:143)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:391)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157)
> at org.apache.hadoop.mapred.Child.main(Child.java:264)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.ClassCastException: org.apache.hadoop.hive.serde2.lazydio.LazyDioInteger cannot be cast to org.apache.hadoop.hive.serde2.lazy.LazyInteger
> at org.apache.hadoop.hive.ql.exec.GroupByOperator.processOp(GroupByOperator.java:737)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
> at org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:83)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
> at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:529)
> ... 9 more
> Caused by: java.lang.ClassCastException: org.apache.hadoop.hive.serde2.lazydio.LazyDioInteger cannot be cast to org.apache.hadoop.hive.serde2.lazy.LazyInteger
> at org.apache.hadoop.hive.serde2.lazy.objectinspector.primitive.LazyIntObjectInspector.copyObject(LazyIntObjectInspector.java:43)
> at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.copyToStandardObject(ObjectInspectorUtils.java:239)
> at org.apache.hadoop.hive.ql.exec.KeyWrapperFactory$ListKeyWrapper.deepCopyElements(KeyWrapperFactory.java:150)
> at org.apache.hadoop.hive.ql.exec.KeyWrapperFactory$ListKeyWrapper.deepCopyElements(KeyWrapperFactory.java:142)
> at org.apache.hadoop.hive.ql.exec.KeyWrapperFactory$ListKeyWrapper.copyKey(KeyWrapperFactory.java:119)
> at org.apache.hadoop.hive.ql.exec.GroupByOperator.processHashAggr(GroupByOperator.java:750)
> at org.apache.hadoop.hive.ql.exec.GroupByOperator.processOp(GroupByOperator.java:722)
> ... 18 more
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira