You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by vinod kumar <vi...@gmail.com> on 2015/07/27 09:04:14 UTC

Functions in Spark SQL

Hi,

May I know how to use the functions mentioned in
http://spark.apache.org/docs/1.4.0/api/scala/index.html#org.apache.spark.sql.functions$
in spark sql?

when I use like

"Select last(column) from tablename" I am getting error like


15/07/27 03:00:00 INFO exec.FunctionRegistry: Unable to lookup UDF in
metastore:
 org.apache.hadoop.hive.ql.metadata.HiveException:
MetaException(message:NoSuchO
bjectException(message:Function default.last does not exist))
java.lang.RuntimeException: Couldn't find function last

Thanks,
Vinod

Re: Functions in Spark SQL

Posted by vinod kumar <vi...@gmail.com>.

Hi,

"Select last(product) from sampleTable"

Spark Version 1.3

-Vinod

On Mon, Jul 27, 2015 at 3:48 AM, fightfate@163.com <fi...@163.com>
wrote:

> Hi, there
>
> I test with sqlContext.sql(select funcName(param1,param2,...) from
> tableName ) just worked fine.
>
> Would you like to paste your test code here ? And which version of Spark
> are u using ?
>
> Best,
> Sun.
>
> ------------------------------
> fightfate@163.com
>
>
> *From:* vinod kumar <vi...@gmail.com>
> *Date:* 2015-07-27 15:04
> *To:* User <us...@spark.apache.org>
> *Subject:* Functions in Spark SQL
> Hi,
>
> May I know how to use the functions mentioned in
> http://spark.apache.org/docs/1.4.0/api/scala/index.html#org.apache.spark.sql.functions$
> in spark sql?
>
> when I use like
>
> "Select last(column) from tablename" I am getting error like
>
>
> 15/07/27 03:00:00 INFO exec.FunctionRegistry: Unable to lookup UDF in
> metastore:
>  org.apache.hadoop.hive.ql.metadata.HiveException:
> MetaException(message:NoSuchO
> bjectException(message:Function default.last does not exist))
> java.lang.RuntimeException: Couldn't find function last
>
> Thanks,
> Vinod
>
>
>

Re: Functions in Spark SQL

Posted by "fightfate@163.com" <fi...@163.com>.

Hi, there

I test with sqlContext.sql(select funcName(param1,param2,...) from tableName ) just worked fine. 

Would you like to paste your test code here ? And which version of Spark are u using ? 

Best,
Sun.



fightfate@163.com
 
From: vinod kumar
Date: 2015-07-27 15:04
To: User
Subject: Functions in Spark SQL
Hi,

May I know how to use the functions mentioned in http://spark.apache.org/docs/1.4.0/api/scala/index.html#org.apache.spark.sql.functions$ in spark sql?

when I use like

"Select last(column) from tablename" I am getting error like


15/07/27 03:00:00 INFO exec.FunctionRegistry: Unable to lookup UDF in metastore:
 org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:NoSuchO
bjectException(message:Function default.last does not exist))
java.lang.RuntimeException: Couldn't find function last

Thanks,
Vinod