[jira] [Commented] (SPARK-37865) Spark should not dedup the groupingExpressions when the first child of Union has duplicate columns

["Apache Spark (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-37865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17502724#comment-17502724 ] 

Apache Spark commented on SPARK-37865:
--------------------------------------

User 'karenfeng' has created a pull request for this issue:
https://github.com/apache/spark/pull/35760

> Spark should not dedup the groupingExpressions when the first child of Union has duplicate columns
> --------------------------------------------------------------------------------------------------
>
>                 Key: SPARK-37865
>                 URL: https://issues.apache.org/jira/browse/SPARK-37865
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 3.3.0
>            Reporter: Chao Gao
>            Priority: Major
>
> When the first child of Union has duplicate columns like select a, a from t1 union select a, b from t2, spark only use the first column to aggregate the results, which would make the results incorrect, and this behavior is inconsistent with other engines like PostgreSQL, MySQL. We could alias the attribute of the first child of union to resolve this, or you could argue that this is the feature of Spark SQL.
> sample query:
> select
> a,
> a
> from values (1, 1), (1, 2) as t1(a, b)
> UNION
> SELECT
> a,
> b
> from values (1, 1), (1, 2) as t2(a, b)
> result is
> (1,1)
> result from PostgreSQL and MySQL
> (1,1)
> (1,2)



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org