You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Matt McCline (JIRA)" <ji...@apache.org> on 2014/08/04 23:38:14 UTC
[jira] [Commented] (HIVE-7426) ClassCastException: ...IntWritable
cannot be cast to ...Text involving
ql.udf.generic.GenericUDFBasePad.evaluate
[ https://issues.apache.org/jira/browse/HIVE-7426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14085334#comment-14085334 ]
Matt McCline commented on HIVE-7426:
------------------------------------
From: Jason Dere
12:52 AM (13 hours ago)
Looking at the stack trace and code, GenericUDFBasePad (line 65) expects the 3 UDF arguments to be of type Text, IntWritable, Text, respectively. However, the argument checking done in initialize() only enforces that all 3 arg types are text/int, but it doesn't check that arg 1 is text, arg 2 is int, arg3 is text. So it looks like the query passed in an int value to arg1, and the converter returned an IntWritable where it expected a Text. The generic checkArguments() didn't do its job here - probably needed to have an additional param specifying what the expected type should be.
What it probably should have done is allow arg1/arg3 to be almost any type, but to convert the value to string/text during evaluate(). That would allow the int passed in arg1 to be implicitly cast to string, and the query as written would work. Or if we want to be strict with the types, Hive could actually throw an error when checking the param types in initialize(). In this case then the user would have to explicitly cast the integer value to string such as LPAD(CAST(<integer_expression> AS STRING), 2, 0).
> ClassCastException: ...IntWritable cannot be cast to ...Text involving ql.udf.generic.GenericUDFBasePad.evaluate
> ----------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-7426
> URL: https://issues.apache.org/jira/browse/HIVE-7426
> Project: Hive
> Issue Type: Bug
> Reporter: Matt McCline
> Attachments: TestWithORC.zip, fail_366.sql, fail_750.sql, fail_856.sql
>
>
> One of several found by Raj Bains.
> M/R or Tez.
> Query does not vectorize, so this is not vector related.
> Query:
> {code}
> SELECT `Calcs`.`datetime0` AS `none_datetime0_ok`, `Calcs`.`int1` AS `none_int1_ok`, `Calcs`.`key` AS `none_key_nk`, CASE WHEN (`Calcs`.`datetime0` IS NOT NULL AND `Calcs`.`int1` IS NOT NULL) THEN FROM_UNIXTIME(UNIX_TIMESTAMP(CONCAT((YEAR(`Calcs`.`datetime0`)+FLOOR((MONTH(`Calcs`.`datetime0`)+`Calcs`.`int1`)/12)), CONCAT('-', CONCAT(LPAD(PMOD(MONTH(`Calcs`.`datetime0`)+`Calcs`.`int1`, 12), 2, '0'), SUBSTR(`Calcs`.`datetime0`, 8)))), SUBSTR('yyyy-MM-dd HH:mm:ss',0,LENGTH(`Calcs`.`datetime0`))), 'yyyy-MM-dd HH:mm:ss') END AS `none_z_dateadd_month_ok` FROM `default`.`testv1_Calcs` `Calcs` GROUP BY `Calcs`.`datetime0`, `Calcs`.`int1`, `Calcs`.`key`, CASE WHEN (`Calcs`.`datetime0` IS NOT NULL AND `Calcs`.`int1` IS NOT NULL) THEN FROM_UNIXTIME(UNIX_TIMESTAMP(CONCAT((YEAR(`Calcs`.`datetime0`)+FLOOR((MONTH(`Calcs`.`datetime0`)+`Calcs`.`int1`)/12)), CONCAT('-', CONCAT(LPAD(PMOD(MONTH(`Calcs`.`datetime0`)+`Calcs`.`int1`, 12), 2, '0'), SUBSTR(`Calcs`.`datetime0`, 8)))), SUBSTR('yyyy-MM-dd HH:mm:ss',0,LENGTH(`Calcs`.`datetime0`))), 'yyyy-MM-dd HH:mm:ss') END ;
> {code}
> Stack Trace:
> {code}
> Caused by: java.lang.ClassCastException: org.apache.hadoop.io.IntWritable cannot be cast to org.apache.hadoop.io.Text
> at org.apache.hadoop.hive.ql.udf.generic.GenericUDFBasePad.evaluate(GenericUDFBasePad.java:65)
> at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator._evaluate(ExprNodeGenericFuncEvaluator.java:166)
> at org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:77)
> at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator$DeferredExprObject.get(ExprNodeGenericFuncEvaluator.java:77)
> at org.apache.hadoop.hive.ql.udf.generic.GenericUDFConcat.stringEvaluate(GenericUDFConcat.java:189)
> at org.apache.hadoop.hive.ql.udf.generic.GenericUDFConcat.evaluate(GenericUDFConcat.java:159)
> at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator._evaluate(ExprNodeGenericFuncEvaluator.java:166)
> at org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:77)
> at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator$DeferredExprObject.get(ExprNodeGenericFuncEvaluator.java:77)
> at org.apache.hadoop.hive.ql.udf.generic.GenericUDFConcat.stringEvaluate(GenericUDFConcat.java:189)
> at org.apache.hadoop.hive.ql.udf.generic.GenericUDFConcat.evaluate(GenericUDFConcat.java:159)
> at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator._evaluate(ExprNodeGenericFuncEvaluator.java:166)
> at org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:77)
> at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator$DeferredExprObject.get(ExprNodeGenericFuncEvaluator.java:77)
> at org.apache.hadoop.hive.ql.udf.generic.GenericUDFConcat.stringEvaluate(GenericUDFConcat.java:189)
> at org.apache.hadoop.hive.ql.udf.generic.GenericUDFConcat.evaluate(GenericUDFConcat.java:159)
> at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator._evaluate(ExprNodeGenericFuncEvaluator.java:166)
> at org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:77)
> at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator$DeferredExprObject.get(ExprNodeGenericFuncEvaluator.java:77)
> at org.apache.hadoop.hive.ql.udf.generic.GenericUDFToUnixTimeStamp.evaluate(GenericUDFToUnixTimeStamp.java:121)
> at org.apache.hadoop.hive.ql.udf.generic.GenericUDFUnixTimeStamp.evaluate(GenericUDFUnixTimeStamp.java:52)
> at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator._evaluate(ExprNodeGenericFuncEvaluator.java:166)
> at org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:77)
> at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator$DeferredExprObject.get(ExprNodeGenericFuncEvaluator.java:77)
> at org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.evaluate(GenericUDFBridge.java:177)
> at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator._evaluate(ExprNodeGenericFuncEvaluator.java:166)
> at org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:77)
> at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator$DeferredExprObject.get(ExprNodeGenericFuncEvaluator.java:77)
> at org.apache.hadoop.hive.ql.udf.generic.GenericUDFWhen.evaluate(GenericUDFWhen.java:78)
> at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator._evaluate(ExprNodeGenericFuncEvaluator.java:166)
> at org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:77)
> at org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:65)
> at org.apache.hadoop.hive.ql.exec.KeyWrapperFactory$ListKeyWrapper.getNewKey(KeyWrapperFactory.java:113)
> at org.apache.hadoop.hive.ql.exec.GroupByOperator.processOp(GroupByOperator.java:778)
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)