You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Adrian Wang (Jira)" <ji...@apache.org> on 2019/12/06 01:39:00 UTC
[jira] [Resolved] (HIVE-22108) Cannot use structured field in a
window function for a text table
[ https://issues.apache.org/jira/browse/HIVE-22108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Adrian Wang resolved HIVE-22108.
--------------------------------
Resolution: Duplicate
Already fixed in HIVE-21104
> Cannot use structured field in a window function for a text table
> -----------------------------------------------------------------
>
> Key: HIVE-22108
> URL: https://issues.apache.org/jira/browse/HIVE-22108
> Project: Hive
> Issue Type: Bug
> Affects Versions: 3.0.0
> Reporter: Xianyin Xin
> Priority: Major
>
> Sample text which has a map as its field:
> {code:java}
> E1719F13B8213BA7EE1694B0D2DC838B,1563490705041,detailtype:wt|topansweruuid:|questionuuid:57291db409ee405cb572d347babd8416
> E8EB4433DB8F4CAB84FCDA769E2AE7BA,1563490701704,detailtype:wt|topansweruuid:|questionuuid:b26abb023eae4efc982924d35aad1f57{code}
> Create table,
> {code:java}
> CREATE TABLE temp_table(
> user_id string,
> ct STRING,
> ep MAP<STRING,STRING>
> )
> ROW FORMAT DELIMITED FIELDS TERMINATED BY ','
> COLLECTION ITEMS TERMINATED BY '|'
> MAP KEYS TERMINATED BY ":"
> STORED AS TEXTFILE
> LOCATION '/user/test/temp_table'; {code}
> Execute query,
> {code:java}
> SELECT lag(ep, 1) over (partition by user_id order by ct) as lag_ep
> from temp_table;{code}
> It thows,
> {code:java}
> 2019-08-14T10:46:01,626 INFO [pool-11-thread-1] exec.FileSinkOperator: Using serializer : class org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe[[[B@d44c263]:[_col0]:[map<string,string>]] and formatter : org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat@751a939a
> 2019-08-14T10:46:01,663 ERROR [pool-11-thread-1] ExecReducer: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row (tag=0) {"key":{"reducesinkkey0":"04E55B1D60F5D70807BF224997623188","reducesinkkey1":"1563490210162"},"value":{"_col0":{"detailtype":"wt","topansweruuid":"","questionuuid":"3efc91897aec41249d39e2d2a4a6c0e5"}}}
> at org.apache.hadoop.hive.ql.exec.mr.ExecReducer.reduce(ExecReducer.java:243)
> at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:444)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:392)
> at org.apache.hadoop.mapred.LocalJobRunner$Job$ReduceTaskRunnable.run(LocalJobRunner.java:319)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.ClassCastException: org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryMap cannot be cast to java.util.Map
> at org.apache.hadoop.hive.serde2.objectinspector.StandardMapObjectInspector.getMap(StandardMapObjectInspector.java:85)
> at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.copyToStandardObject(ObjectInspectorUtils.java:451)
> at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.copyToStandardObject(ObjectInspectorUtils.java:372)
> at org.apache.hadoop.hive.ql.udf.generic.GenericUDAFLeadLag$GenericUDAFLeadLagEvaluator.iterate(GenericUDAFLeadLag.java:156)
> at org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.aggregate(GenericUDAFEvaluator.java:213)
> at org.apache.hadoop.hive.ql.udf.ptf.WindowingTableFunction.processRow(WindowingTableFunction.java:407)
> at org.apache.hadoop.hive.ql.exec.PTFOperator$PTFInvocation.processRow(PTFOperator.java:325)
> at org.apache.hadoop.hive.ql.exec.PTFOperator.process(PTFOperator.java:139)
> at org.apache.hadoop.hive.ql.exec.Operator.baseForward(Operator.java:995)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:941)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:928)
> at org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:95)
> at org.apache.hadoop.hive.ql.exec.mr.ExecReducer.reduce(ExecReducer.java:234){code}
> However, simple select is OK:
> {code:java}
> select ep['detailtype'] from temp_table;{code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)