You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by vishpatel <vi...@gmail.com> on 2015/02/05 02:41:39 UTC

Is there a way to access Hive UDFs in a HiveContext?

I'm trying to access a permanent Hive UDF,

scala> val data = hc.sql("select func.md5(some_string) from some_table")
data: org.apache.spark.sql.SchemaRDD =
SchemaRDD[19] at RDD at SchemaRDD.scala:108
== Query Plan ==
== Physical Plan ==
java.lang.RuntimeException: Couldn't find function func.md5

Version: Spark 1.2 on CDH 5.3.





--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Is-there-a-way-to-access-Hive-UDFs-in-a-HiveContext-tp21510.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org


Re: Is there a way to access Hive UDFs in a HiveContext?

Posted by jamborta <ja...@gmail.com>.
Hi,

My guess is that Spark is not picking up the jar where the function is
stored. You might have to add it to sparkcontext or the classpath manually.

You can also register the function

hc.registerFunction("myfunct", myfunct)

then use it in the query.



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Is-there-a-way-to-access-Hive-UDFs-in-a-HiveContext-tp21510p21527.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org