Using Hive UDTF in SparkSQL

[Lokesh Yadav <lo...@gmail.com>]

Hello

I am trying to use a Hive UDTF function in spark SQL. But somehow its not
working for me as intended and I am not able to understand the behavior.

When I try to register a function like this:
create temporary function SampleUDTF_01 as
'com.fl.experiments.sparkHive.SampleUDTF' using JAR
'hdfs:///user/root/sparkHive-1.0.0.jar';
It successfully registers the function, but gives me a 'not a registered
function' error when I try to run that function. Also it doesn't show up in
the list when I do a 'show functions'.

Another case:
When I try to register the same function as a temporary function using a
local jar (the hdfs path doesn't work with temporary function, that is
weird too), it registers, and I am able to successfully run that function
as well. Another weird thing is that I am not able to drop that function
using the 'drop function ...' statement. This the functions shows up in the
function registry.

I am stuck with this, any help would be really appreciated.
Thanks

Regards,
Lokesh Yadav

Re: Using Hive UDTF in SparkSQL

[Davies Liu <da...@databricks.com>]

Could you file a JIRA for this bug?

On Thu, Oct 27, 2016 at 3:05 AM, Lokesh Yadav
<lo...@gmail.com> wrote:
> Hello
>
> I am trying to use a Hive UDTF function in spark SQL. But somehow its not
> working for me as intended and I am not able to understand the behavior.
>
> When I try to register a function like this:
> create temporary function SampleUDTF_01 as
> 'com.fl.experiments.sparkHive.SampleUDTF' using JAR
> 'hdfs:///user/root/sparkHive-1.0.0.jar';
> It successfully registers the function, but gives me a 'not a registered
> function' error when I try to run that function. Also it doesn't show up in
> the list when I do a 'show functions'.
>
> Another case:
> When I try to register the same function as a temporary function using a
> local jar (the hdfs path doesn't work with temporary function, that is weird
> too), it registers, and I am able to successfully run that function as well.
> Another weird thing is that I am not able to drop that function using the
> 'drop function ...' statement. This the functions shows up in the function
> registry.
>
> I am stuck with this, any help would be really appreciated.
> Thanks
>
> Regards,
> Lokesh Yadav

---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org