[jira] [Created] (SPARK-18861) Spark-SQL unconsistent behavior with "struct" expressions

["Ohad Raviv (JIRA)" <ji...@apache.org>]

Ohad Raviv created SPARK-18861:
----------------------------------

             Summary: Spark-SQL unconsistent behavior with "struct" expressions
                 Key: SPARK-18861
                 URL: https://issues.apache.org/jira/browse/SPARK-18861
             Project: Spark
          Issue Type: Bug
          Components: SQL
    Affects Versions: 2.0.0
            Reporter: Ohad Raviv


We are getting strangly inconsistent behavior with expressions involving "struct". Let's start with this simple table:
{quote}
Seq((1, 2, 3), (2, 3, 4)).toDF("a", "b", "c").createOrReplaceTempView("t1")
sql("desc t1").show()
{quote}
Then we get this DF:
{quote}
|col_name|data_type|comment|
|       a|      int|       |
|       b|      int|       |
|       c|      int|       |
{quote}
Now, although we can clearly see that all the fields are of type int, we we run:
{quote}
sql("SELECT case when a>b then struct(a,b) else struct(c,c) end from t1")
{quote}
we get this error:
{quote}
org.apache.spark.sql.AnalysisException: cannot resolve 'CASE WHEN (t1.`a` > t1.`b`) THEN struct(t1.`a`, t1.`b`) ELSE struct(t1.`c`, t1.`c`) END' due to data type mismatch: THEN and ELSE expressions should all be same type or coercible to a common type; line 1 pos 7
{quote}
if we try this:
{quote}
sql("SELECT case when a>b then struct(cast(a as int), cast(b as int)) else struct(cast(c as int), cast(c as int)) end from t1")
{quote}
we get another exception:
{quote}
requirement failed: Unresolved attributes found when constructing LocalRelation.
java.lang.IllegalArgumentException: requirement failed: Unresolved attributes found when constructing LocalRelation.
	at scala.Predef$.require(Predef.scala:224)
	at org.apache.spark.sql.catalyst.plans.logical.LocalRelation.<init>(LocalRelation.scala:49)
{quote}
However, these do work:
{quote}
sql("SELECT case when a>b then struct(cast(a as double), cast(b as double)) else struct(cast(c as double), cast(c as double)) end from t1")
sql("SELECT case when a>b then struct(cast(a as string), cast(b as string)) else struct(cast(c as string), cast(c as string)) end from t1")
{quote}

any ideas?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org