You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Josh Rosen (JIRA)" <ji...@apache.org> on 2016/08/17 20:11:21 UTC
[jira] [Created] (SPARK-17114) Adding a 'GROUP BY 1' where first
column is literal results in wrong answer
Josh Rosen created SPARK-17114:
----------------------------------
Summary: Adding a 'GROUP BY 1' where first column is literal results in wrong answer
Key: SPARK-17114
URL: https://issues.apache.org/jira/browse/SPARK-17114
Project: Spark
Issue Type: Bug
Affects Versions: 2.0.0, 1.6.2
Reporter: Josh Rosen
Consider the following example:
{code}
sc.parallelize(Seq(128, 256)).toDF("int_col").registerTempTable("mytable")
// The following query should return an empty result set because the `IN` filter condition is always false for this single-row table.
val withoutGroupBy = sqlContext.sql("""
SELECT 'foo'
FROM mytable
WHERE int_col == 0
""")
assert(withoutGroupBy.collect().isEmpty, "original query returned wrong answer")
// After adding a 'GROUP BY 1' the query result should still be empty because we'd be grouping an empty table:
val withGroupBy = sqlContext.sql("""
SELECT 'foo'
FROM mytable
WHERE int_col == 0
GROUP BY 1
""")
assert(withGroupBy.collect().isEmpty, "adding GROUP BY resulted in wrong answer")
{code}
Here, this fails the second assertion by returning a single row. It appears that running {{group by 1}} where column 1 is a constant causes filter conditions to be ignored.
Both PostgreSQL and SQLite return empty result sets for the query containing the {{GROUP BY}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org