[jira] [Commented] (SPARK-6382) withUDF(...) {...} for supporting temporary UDF definitions in the scope

["Xiao Li (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-6382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15554375#comment-15554375 ] 

Xiao Li commented on SPARK-6382:
--------------------------------

This becomes invalid in the latest master branch. Please reopen it if you have any question. Thanks!

> withUDF(...) {...} for supporting temporary UDF definitions in the scope
> ------------------------------------------------------------------------
>
>                 Key: SPARK-6382
>                 URL: https://issues.apache.org/jira/browse/SPARK-6382
>             Project: Spark
>          Issue Type: Improvement
>          Components: SQL
>    Affects Versions: 1.3.0, 1.3.1
>            Reporter: Jianshi Huang
>
> Currently the scope of UDF registration is global. It's unsuitable for libraries that are built on top of DataFrame, as many operations has to be done by registering a UDF first.
> Please provide a way for binding temporary UDFs.
> e.g.
> {code}
> withUDF(("merge_map", (m1: Map[String, Double], m2: Map[String, Double]) => m2 ++ m2),
>     ...) {
>   sql("select merge_map(d1.map, d2.map) from d1, d2 where d1.id = d2.id")
> }
> {code}
> Also UDF registry is a mutable Hashmap, refactoring it to a immutable one makes more sense.
> Jianshi



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org