You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Cheng Hao (JIRA)" <ji...@apache.org> on 2014/09/12 07:20:33 UTC
[jira] [Commented] (SPARK-3501) Hive SimpleUDF will create
duplicated type cast which cause exception in constant folding
[ https://issues.apache.org/jira/browse/SPARK-3501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14131119#comment-14131119 ]
Cheng Hao commented on SPARK-3501:
----------------------------------
https://github.com/apache/spark/pull/2368
> Hive SimpleUDF will create duplicated type cast which cause exception in constant folding
> -----------------------------------------------------------------------------------------
>
> Key: SPARK-3501
> URL: https://issues.apache.org/jira/browse/SPARK-3501
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Reporter: Cheng Hao
> Priority: Minor
>
> When do the query like:
> select datediff(cast(value as timestamp), cast('2002-03-21 00:00:00' as timestamp)) from src;
> SparkSQL will raise exception:
> {panel}
> [info] - Cast Timestamp to Timestamp in UDF *** FAILED ***
> [info] scala.MatchError: TimestampType (of class org.apache.spark.sql.catalyst.types.TimestampType$)
> [info] at org.apache.spark.sql.catalyst.expressions.Cast.castToTimestamp(Cast.scala:77)
> [info] at org.apache.spark.sql.catalyst.expressions.Cast.cast$lzycompute(Cast.scala:251)
> [info] at org.apache.spark.sql.catalyst.expressions.Cast.cast(Cast.scala:247)
> [info] at org.apache.spark.sql.catalyst.expressions.Cast.eval(Cast.scala:263)
> [info] at org.apache.spark.sql.catalyst.optimizer.ConstantFolding$$anonfun$apply$5$$anonfun$applyOrElse$2.applyOrElse(Optimizer.scala:217)
> [info] at org.apache.spark.sql.catalyst.optimizer.ConstantFolding$$anonfun$apply$5$$anonfun$applyOrElse$2.applyOrElse(Optimizer.scala:210)
> [info] at org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:144)
> [info] at org.apache.spark.sql.catalyst.trees.TreeNode$$anonfun$4$$anonfun$apply$2.apply(TreeNode.scala:180)
> [info] at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
> [info] at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
> {panel}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org