You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Udbhav Agrawal (JIRA)" <ji...@apache.org> on 2018/12/27 14:10:00 UTC
[jira] [Commented] (SPARK-26454) IllegegalArgument Exception is
Thrown while creating new UDF with JAR
[ https://issues.apache.org/jira/browse/SPARK-26454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16729628#comment-16729628 ]
Udbhav Agrawal commented on SPARK-26454:
----------------------------------------
[~dongjoon] [~cloud_fan]
Creating the UDF for the first time using Jar options, HDFS path is converted to local path and JAR is added to that path.
Now, Creating the function second time will check if that Jar is present and associated with any other path, it will Throw an IllegalArgumentException, and continue creating the function.
> IllegegalArgument Exception is Thrown while creating new UDF with JAR
> ---------------------------------------------------------------------
>
> Key: SPARK-26454
> URL: https://issues.apache.org/jira/browse/SPARK-26454
> Project: Spark
> Issue Type: Bug
> Components: Spark Shell
> Affects Versions: 2.3.2
> Reporter: Udbhav Agrawal
> Priority: Major
>
> 【Test step】：
> 1.launch spark-shell
> 2. set role admin;
> 3. create new function
> CREATE FUNCTION Func AS 'org.apache.hadoop.hive.ql.udf.generic.GenericUDFLastDayTest' USING JAR 'hdfs:///tmp/super_udf/two_udfs.jar'
> 4. Do select on the function
> sql("select Func('2018-03-09')").show()
> 5.Create new UDF with same JAR
> sql("CREATE FUNCTION newFunc AS 'org.apache.hadoop.hive.ql.udf.generic.GenericUDFLastDayTest' USING JAR 'hdfs:///tmp/super_udf/two_udfs.jar'")
> 6. Do select on the new function created.
> sql("select newFunc ('2018-03-09')").show()
> 【Output】:
> Function is getting created but illegal argument exception is thrown , select provides result but with illegal argument exception.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org