[jira] [Commented] (SPARK-17683) Support ArrayType in Literal.apply

["Takeshi Yamamuro (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-17683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15664088#comment-15664088 ] 

Takeshi Yamamuro commented on SPARK-17683:
------------------------------------------

[~rxin] I revisited this issue; if it's possible to replace `functions.lit` while keeping back-compatibility, we can support Seq[_]
https://github.com/apache/spark/compare/master...maropu:Literal2
{code}

scala> val df1 = Seq((1, 0), (2, 3)).toDF("a", "b")
df1: org.apache.spark.sql.DataFrame = [a: int, b: int]

scala> df1.withColumn("d", lit2(Seq(1, 2)))
res9: org.apache.spark.sql.DataFrame = [a: int, b: int ... 1 more field]

scala> val df2 = df1.withColumn("d", lit2(Seq(1, 2)))
df2: org.apache.spark.sql.DataFrame = [a: int, b: int ... 1 more field]

scala> df2.printSchema
root
 |-- a: integer (nullable = false)
 |-- b: integer (nullable = false)
 |-- d: array (nullable = false)
 |    |-- element: integer (containsNull = false)

scala> df2.show
+---+---+------+
|  a|  b|     d|
+---+---+------+
|  1|  0|[1, 2]|
|  2|  3|[1, 2]|
+---+---+------+
{code}

This also can support Map;
{code}
scala> val df3 = df.withColumn("d", lit2(Map(1 -> 2)))
df3: org.apache.spark.sql.DataFrame = [a: int, b: int ... 1 more field]

scala> df3.printSchema
root
 |-- a: integer (nullable = false)
 |-- b: integer (nullable = false)
 |-- d: map (nullable = false)
 |    |-- key: integer
 |    |-- value: integer (valueContainsNull = false)

scala> df3.show
+---+---+-----------+
|  a|  b|          d|
+---+---+-----------+
|  1|  0|Map(1 -> 2)|
|  2|  3|Map(1 -> 2)|
+---+---+-----------+
{code}

> Support ArrayType in Literal.apply
> ----------------------------------
>
>                 Key: SPARK-17683
>                 URL: https://issues.apache.org/jira/browse/SPARK-17683
>             Project: Spark
>          Issue Type: Improvement
>          Components: SQL
>    Affects Versions: 2.0.0
>            Reporter: Takeshi Yamamuro
>            Assignee: Takeshi Yamamuro
>             Fix For: 2.1.0
>
>
> This ticket comes from http://apache-spark-developers-list.1001551.n3.nabble.com/ArrayType-support-in-Spark-SQL-td19063.html



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org