[jira] [Commented] (SPARK-6376) Subqueries are thrown away too early in dataframes

["Reynold Xin (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-6376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14493648#comment-14493648 ] 

Reynold Xin commented on SPARK-6376:
------------------------------------

Note that SPARK-6865 will break this since $"x.str" will refer to a column named "x.str", rather than a column named "str" in table "x".


> Subqueries are thrown away too early in dataframes
> --------------------------------------------------
>
>                 Key: SPARK-6376
>                 URL: https://issues.apache.org/jira/browse/SPARK-6376
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>            Reporter: Michael Armbrust
>            Assignee: Michael Armbrust
>            Priority: Critical
>             Fix For: 1.3.1, 1.4.0
>
>
> Because we throw away aliases as we construct the query plan, you can't reference them later.  For example, this query fails:
> {code}
>   test("self join with aliases") {
>     val df = Seq(1,2,3).map(i => (i, i.toString)).toDF("int", "str")
>     checkAnswer(
>       df.as('x).join(df.as('y), $"x.str" === $"y.str").groupBy("x.str").count(),
>       Row("1", 1) :: Row("2", 1) :: Row("3", 1) :: Nil)
>   }
> {code}
> {code}
> [info]   org.apache.spark.sql.AnalysisException: Cannot resolve column name "x.str" among (int, str, int, str);
> [info]   at org.apache.spark.sql.DataFrame$$anonfun$resolve$1.apply(DataFrame.scala:162)
> [info]   at org.apache.spark.sql.DataFrame$$anonfun$resolve$1.apply(DataFrame.scala:162)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org