[jira] [Commented] (HIVE-2540) LATERAL VIEW with EXPLODE produces ConcurrentModificationException

["Lars Francke (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-2540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13976820#comment-13976820 ] 

Lars Francke commented on HIVE-2540:
------------------------------------

We're still seeing this issue with Hive 0.12 (slightly patched as it's the one from CDH5):

{quote}
2014-04-22 16:18:50,304 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {"product_id":140719,"active":null,"avg_product_price":null,"base_cats":[[1247,818,821,902,905],[1247,818,821,902,906],[1247,818,821,902,909]]}
	at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:175)
	at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {"product_id":140719,"active":null,"avg_product_price":null,"base_cats":[[1247,818,821,902,905],[1247,818,821,902,906],[1247,818,821,902,909]]}
	at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:529)
	at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:157)
	... 8 more
Caused by: java.util.ConcurrentModificationException
	at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:859)
	at java.util.ArrayList$Itr.next(ArrayList.java:831)
	at org.apache.hadoop.hive.ql.udf.generic.GenericUDTFExplode.process(GenericUDTFExplode.java:92)
	at org.apache.hadoop.hive.ql.exec.UDTFOperator.processOp(UDTFOperator.java:113)
	at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:504)
	at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:844)
	at org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:87)
	at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:504)
	at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:844)
	at org.apache.hadoop.hive.ql.exec.LateralViewForwardOperator.processOp(LateralViewForwardOperator.java:37)
	at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:504)
	at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:844)
	at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:91)
	at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:504)
	at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:844)
	at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:519)
	... 9 more
{quote}

I haven't had time to check whether this patch is actually included but I'd expect it to be. Shall I open a new issue?

> LATERAL VIEW with EXPLODE produces ConcurrentModificationException
> ------------------------------------------------------------------
>
>                 Key: HIVE-2540
>                 URL: https://issues.apache.org/jira/browse/HIVE-2540
>             Project: Hive
>          Issue Type: Bug
>          Components: Query Processor
>    Affects Versions: 0.7.1, 0.9.0
>            Reporter: David Phillips
>            Assignee: Navis
>             Fix For: 0.10.0
>
>         Attachments: ASF.LICENSE.NOT.GRANTED--HIVE-2540.D2805.1.patch, ASF.LICENSE.NOT.GRANTED--HIVE-2540.D2805.2.patch, ASF.LICENSE.NOT.GRANTED--HIVE-2540.D2805.3.patch
>
>
> The following produces {{ConcurrentModificationException}} on the {{for}} loop inside EXPLODE:
> {code}
> create table foo as select array(1, 2) a from src limit 1;
> select a, x.b from foo lateral view explode(a) x as b;
> {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)