You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Ankita Bakshi <an...@gmail.com> on 2014/09/24 00:25:17 UTC
Issues after upgrading to hive0.13 from hive0.12
1. The following query works in Hive0.12 but not is hive0.13. Note, in
hive0.12 the following query will not compile if we remove one of the Day
column from group by.
SELECT DAY, daypart, 'DAY' AS DIMENSION, `day` AS VALUE, SUM(`total`) AS T
FROM H_ANKITEST GROUP BY DAY, `day` , daypart
FAILED: SemanticException [Error 10021]: Line 2:41 Repeated key in GROUP
BY 'day'
2. UNION ALL fails with the IndexOutOfBoundsException exception. Note. this
is reproducible when we have investigationId column which is an INT type in
UNION ALL, but it succeeds for other columns which are of INT type.
*Query*:
SELECT daypart, DAY, 'Z_AIRPLANEMODE' AS CUBENAME, DIMENSION, VALUE
FROM (
SELECT DAY, daypart, 'WEEK' AS DIMENSION, `week` AS VALUE,
SUM(`total`) AS T
FROM H_ANKITEST GROUP BY DAY, daypart, `week`
ORDER BY T DESC LIMIT 1000
UNION ALL
SELECT DAY, daypart, 'INVESTIGATIONID' AS DIMENSION, `investigationId`
AS VALUE, SUM(`total`) AS T
FROM H_ANKITEST GROUP BY DAY, daypart, `investigationId`
ORDER BY T DESC LIMIT 1000
)
OUTER
*Plan for the failed task in Hive0.13:*
2014-09-23 21:51:15,723 INFO org.apache.hadoop.hive.ql.exec.MapOperator:*
dump TS struct<_col1:string,_col0:string,_col2:string,_col3:string> // col4
is missing*
<MAP>Id =5
<Children>
<TS>Id =0
<Children>
<UNION>Id =1
<Children>
<SEL>Id =2
<Children>
<FS>Id =3
<Children>
<\Children>
<Parent>Id = 2 null<\Parent>
<\FS>
<\Children>
<Parent>Id = 1 null<\Parent>
<\SEL>
<\Children>
<Parent>Id = 4
<TS>Id =4
<Children>null
<\Children>
<Parent>Id = 5 null<\Parent>
<\TS>Id = 0 null<\Parent>
<\UNION>
<\Children>
<Parent>Id = 5 null<\Parent>
<\TS>
<\Children>
<Parent><\Parent>
<\MAP>
*Plan for the successful task in Hive0.12*
2014-09-23 22:05:22,022 INFO org.apache.hadoop.hive.ql.exec.MapOperator:
dump TS struct<_col0:string,_col1:string,_col2:string,_col3:string,
*_col4:bigint*>
<MAP>Id =5
<Children>
<TS>Id =4
<Children>
<UNION>Id =2
<Children>
<SEL>Id =1
<Children>
<FS>Id =0
<Parent>Id = 1 null<\Parent>
<\FS>
<\Children>
<Parent>Id = 2 null<\Parent>
<\SEL>
<\Children>
<Parent>Id = 3
<TS>Id =3
<Children>null
<\Children>
<Parent>Id = 5 null<\Parent>
<\TS>Id = 4 null<\Parent>
<\UNION>
<\Children>
<Parent>Id = 5 null<\Parent>
<\TS>
<\Children>
<\MAP>
*Exception*:
java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:75)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:413)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:332)
at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
at org.apache.hadoop.mapred.Child.main(Child.java:262)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
... 9 more
Caused by: java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:75)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
... 14 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
... 17 more
Caused by: java.lang.RuntimeException: Map operator initialization failed
at
org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:154)
... 22 more
Caused by: java.lang.IndexOutOfBoundsException: Index: 4, Size: 4
at java.util.ArrayList.rangeCheck(ArrayList.java:635)
at java.util.ArrayList.get(ArrayList.java:411)
at
org.apache.hadoop.hive.ql.exec.UnionOperator.initializeOp(UnionOperator.java:83)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:460)
at
org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:416)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:189)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
at
org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:443)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
at
org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:133)
Thanks,
Ankita