[jira] [Commented] (SPARK-2693) Support for UDAF Hive Aggregates like PERCENTILE

["Ravindra Pesala (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14110707#comment-14110707 ] 

Ravindra Pesala commented on SPARK-2693:
----------------------------------------

UDAF is deprecated in HIve, Though there can be few functions which could have implemented using this interface. We can support the same in spark for backward compatability. 
As you mentioned supporting UDAF in spark requires to write a wrapper.
*Please assign it to me.*

> Support for UDAF Hive Aggregates like PERCENTILE
> ------------------------------------------------
>
>                 Key: SPARK-2693
>                 URL: https://issues.apache.org/jira/browse/SPARK-2693
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>            Reporter: Michael Armbrust
>
> {code}
> SELECT MIN(field1), MAX(field2), AVG(field3), PERCENTILE(field4), year,month,day FROM  raw_data_table  GROUP BY year, month, day
> MIN, MAX and AVG functions work fine for me, but with PERCENTILE, I get an error as shown below.
> Exception in thread "main" java.lang.RuntimeException: No handler for udf class org.apache.hadoop.hive.ql.udf.UDAFPercentile
>         at scala.sys.package$.error(package.scala:27)
>         at org.apache.spark.sql.hive.HiveFunctionRegistry$.lookupFunction(hiveUdfs.scala:69)
>         at org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveFunctions$$anonfun$apply$4$$anonfun$applyOrElse$3.applyOrElse(Analyzer.scala:115)
>         at org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveFunctions$$anonfun$apply$4$$anonfun$applyOrElse$3.applyOrElse(Analyzer.scala:113)
>         at org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:165)
> {code}
> This aggregate extends UDAF, which we don't yet have a wrapper for.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org