You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Cheng Lian (JIRA)" <ji...@apache.org> on 2014/11/03 08:56:34 UTC
[jira] [Created] (SPARK-4202) DSL support for Scala UDF
Cheng Lian created SPARK-4202:
---------------------------------
Summary: DSL support for Scala UDF
Key: SPARK-4202
URL: https://issues.apache.org/jira/browse/SPARK-4202
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 1.1.1
Reporter: Cheng Lian
Using Scala UDF with current DSL API is quite verbose, e.g.:
{code}
case class KeyValue(key: Int, value: String)
val schemaRDD = sc.parallelize(1 to 10).map(i => KeyValue(i, i.toString)).toSchemaRDD
def foo = (a: Int, b: String) => a.toString + b
schemaRDD.select( // SELECT
Star(None), // *,
ScalaUdf( //
foo, // foo(
StringType, //
'key.attr :: 'value.attr :: Nil) // key, value
).collect() // ) FROM ...
{code}
It would be good to add a DSL syntax to simplify UDF invocation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org