You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@toree.apache.org by "Luciano Resende (JIRA)" <ji...@apache.org> on 2017/03/23 17:36:41 UTC
[jira] [Assigned] (TOREE-376) Scala interpreter should make Spark
SQL implicits available
[ https://issues.apache.org/jira/browse/TOREE-376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Luciano Resende reassigned TOREE-376:
-------------------------------------
Assignee: Ryan Blue
> Scala interpreter should make Spark SQL implicits available
> -----------------------------------------------------------
>
> Key: TOREE-376
> URL: https://issues.apache.org/jira/browse/TOREE-376
> Project: TOREE
> Issue Type: Improvement
> Reporter: Ryan Blue
> Assignee: Ryan Blue
>
> The scala interpreter doesn't define Spark SQL functions or implicits that are commonly used and available by default in Spark's shell.
> Relevant test cases:
> {code}
> scala> val c = $"test"
> c: org.apache.spark.sql.ColumnName = test
> scala> val df = Seq((1, "a"), (2, "b")).toDF("id", "data")
> df: org.apache.spark.sql.DataFrame = [id: int, data: string]
> scala> df.withColumn("m", map(lit("id"), $"id")).printSchema
> root
> |-- id: integer (nullable = false)
> |-- data: string (nullable = true)
> |-- m: map (nullable = false)
> | |-- key: string
> | |-- value: integer (valueContainsNull = false)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)