[spark] branch branch-3.3 updated: [SPARK-39887][SQL][FOLLOW-UP] Do not exclude Union's first child attributes when traversing other children in RemoveRedundantAliases

[we...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

wenchen pushed a commit to branch branch-3.3
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/branch-3.3 by this push:
     new 9601be96a86 [SPARK-39887][SQL][FOLLOW-UP] Do not exclude Union's first child attributes when traversing other children in RemoveRedundantAliases
9601be96a86 is described below

commit 9601be96a86eced683e6aa2b772c726eeb231de8
Author: Peter Toth <pt...@cloudera.com>
AuthorDate: Wed Aug 17 14:57:35 2022 +0800

    [SPARK-39887][SQL][FOLLOW-UP] Do not exclude Union's first child attributes when traversing other children in RemoveRedundantAliases
    
    ### What changes were proposed in this pull request?
    Do not exclude `Union`'s first child attributes when traversing other children in `RemoveRedundantAliases`.
    
    ### Why are the changes needed?
    We don't need to exclude those attributes that `Union` inherits from its first child. See discussion here: https://github.com/apache/spark/pull/37496#discussion_r944509115
    
    ### Does this PR introduce _any_ user-facing change?
    No.
    
    ### How was this patch tested?
    Existing UTs.
    
    Closes #37534 from peter-toth/SPARK-39887-keep-attributes-of-unions-first-child-follow-up.
    
    Authored-by: Peter Toth <pt...@cloudera.com>
    Signed-off-by: Wenchen Fan <we...@databricks.com>
    (cherry picked from commit e732232dac420826af269d8cf5efacb52933f59a)
    Signed-off-by: Wenchen Fan <we...@databricks.com>
---
 .../scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala    | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
index 558a67ff5ca..4807824ee71 100644
--- a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
+++ b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
@@ -544,7 +544,7 @@ object RemoveRedundantAliases extends Rule[LogicalPlan] {
         })
         Join(newLeft, newRight, joinType, newCondition, hint)
 
-      case _: Union =>
+      case u: Union =>
         var first = true
         plan.mapChildren { child =>
           if (first) {
@@ -555,7 +555,8 @@ object RemoveRedundantAliases extends Rule[LogicalPlan] {
             // output attributes could return incorrect result.
             removeRedundantAliases(child, excluded ++ child.outputSet)
           } else {
-            removeRedundantAliases(child, excluded)
+            // We don't need to exclude those attributes that `Union` inherits from its first child.
+            removeRedundantAliases(child, excluded -- u.children.head.outputSet)
           }
         }
 


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org