You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2022/08/12 08:06:54 UTC
[GitHub] [spark] peter-toth opened a new pull request, #37491: [SPARK-39887][SQL][3.2] RemoveRedundantAliases should keep aliases that make the output of projection nodes unique
peter-toth opened a new pull request, #37491:
URL: https://github.com/apache/spark/pull/37491
### What changes were proposed in this pull request?
Keep the output attributes of a `Union` node's first child in the `RemoveRedundantAliases` rule to avoid correctness issues.
### Why are the changes needed?
To fix the result of the following query:
```
SELECT a, b AS a FROM (
SELECT a, a AS b FROM (SELECT a FROM VALUES (1) AS t(a))
UNION ALL
SELECT a, b FROM (SELECT a, b FROM VALUES (1, 2) AS t(a, b))
)
```
Before this PR the query returns the incorrect result:
```
+---+---+
| a| a|
+---+---+
| 1| 1|
| 2| 2|
+---+---+
```
After this PR it returns the expected result:
```
+---+---+
| a| a|
+---+---+
| 1| 1|
| 1| 2|
+---+---+
```
### Does this PR introduce _any_ user-facing change?
Yes, fixes a correctness issue.
### How was this patch tested?
Added new UTs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] peter-toth commented on pull request #37491: [SPARK-39887][SQL][3.2] RemoveRedundantAliases should keep aliases that make the output of projection nodes unique
Posted by GitBox <gi...@apache.org>.
peter-toth commented on PR #37491:
URL: https://github.com/apache/spark/pull/37491#issuecomment-1214071440
> Could you re-trigger CI, @peter-toth ? Linter failure is okay, but there exists SQL failure.
>
> <img alt="Screen Shot 2022-08-12 at 4 02 40 PM" width="509" src="https://user-images.githubusercontent.com/9700541/184455020-2fbe3e57-771d-4277-b146-6e8a87a0e7db.png">
Thanks @dongjoon-hyun for checking the results. I've rerun the tests and only `[Linters, licenses, dependencies and documentation generation` failed this time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #37491: [SPARK-39887][SQL][3.2] RemoveRedundantAliases should keep aliases that make the output of projection nodes unique
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on PR #37491:
URL: https://github.com/apache/spark/pull/37491#issuecomment-1213578201
Could you re-trigger CI, @peter-toth ? Linter failure is okay, but there exists SQL failure.
<img width="509" alt="Screen Shot 2022-08-12 at 4 02 40 PM" src="https://user-images.githubusercontent.com/9700541/184455020-2fbe3e57-771d-4277-b146-6e8a87a0e7db.png">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #37491: [SPARK-39887][SQL][3.2] RemoveRedundantAliases should keep aliases that make the output of projection nodes unique
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on PR #37491:
URL: https://github.com/apache/spark/pull/37491#issuecomment-1214478411
Thank you, @peter-toth and @cloud-fan
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] peter-toth commented on pull request #37491: [SPARK-39887][SQL][3.2] RemoveRedundantAliases should keep aliases that make the output of projection nodes unique
Posted by GitBox <gi...@apache.org>.
peter-toth commented on PR #37491:
URL: https://github.com/apache/spark/pull/37491#issuecomment-1213127621
cc @cloud-fan
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun closed pull request #37491: [SPARK-39887][SQL][3.2] RemoveRedundantAliases should keep aliases that make the output of projection nodes unique
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun closed pull request #37491: [SPARK-39887][SQL][3.2] RemoveRedundantAliases should keep aliases that make the output of projection nodes unique
URL: https://github.com/apache/spark/pull/37491
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #37491: [SPARK-39887][SQL][3.2] RemoveRedundantAliases should keep aliases that make the output of projection nodes unique
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on PR #37491:
URL: https://github.com/apache/spark/pull/37491#issuecomment-1214478130
Thank you. I confirmed that, @peter-toth .
Merged to branch-3.2.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org