[spark] branch master updated: [SPARK-43979][SQL][FOLLOW-UP] CollectedMetrics should be treated as the same one for self-join

[ge...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

gengliang pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new 954987f19dc [SPARK-43979][SQL][FOLLOW-UP] CollectedMetrics should be treated as the same one for self-join
954987f19dc is described below

commit 954987f19dca67064268cde023d489eb22d81439
Author: Rui Wang <ru...@databricks.com>
AuthorDate: Tue Jun 27 10:15:02 2023 -0700

    [SPARK-43979][SQL][FOLLOW-UP] CollectedMetrics should be treated as the same one for self-join
    
    ### What changes were proposed in this pull request?
    
    Use `transformUpWithNewOutput` than `resolveOperatorsUpWithNewOutput` to simplify the metrics plan. This is to in case that one plan is analyzed and another one is not analyzed.
    
    ### Why are the changes needed?
    
    To fix the case where we have two CollectedMetrics plan to compare where one is analyzed and another one is not.
    
    ### Does this PR introduce _any_ user-facing change?
    
    No
    ### How was this patch tested?
    
    Existing tests
    
    Closes #41745 from amaliujia/fix_metrics_path.
    
    Authored-by: Rui Wang <ru...@databricks.com>
    Signed-off-by: Gengliang Wang <ge...@apache.org>
---
 .../scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala    | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala
index 649140e466a..7c0e8f1490d 100644
--- a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala
+++ b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala
@@ -1080,7 +1080,7 @@ trait CheckAnalysis extends PredicateHelper with LookupCatalog with QueryErrorsB
    * duplicates metric definition.
    */
   private def simplifyPlanForCollectedMetrics(plan: LogicalPlan): LogicalPlan = {
-    plan.resolveOperatorsUpWithNewOutput {
+    plan.transformUpWithNewOutput {
       case p: Project if p.projectList.size == p.child.output.size =>
         val assignExprIdOnly = p.projectList.zip(p.child.output).forall {
           case (left: Alias, right: Attribute) =>


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org