[GitHub] [spark] ulysses-you commented on a diff in pull request #39624: [SPARK-42101][SQL] Introduce Materializable and MaterializableQueryStage for AQE framework

["ulysses-you (via GitHub)" <gi...@apache.org>]

ulysses-you commented on code in PR #39624:
URL: https://github.com/apache/spark/pull/39624#discussion_r1122539330


##########
sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/AdaptiveSparkPlanExec.scala:
##########
@@ -220,10 +220,15 @@ case class AdaptiveSparkPlanExec(
   }
 
   private def getExecutionId: Option[Long] = {
-    // If the `QueryExecution` does not match the current execution ID, it means the execution ID
-    // belongs to another (parent) query, and we should not call update UI in this query.
     Option(context.session.sparkContext.getLocalProperty(SQLExecution.EXECUTION_ID_KEY))
-      .map(_.toLong).filter(SQLExecution.getQueryExecution(_) eq context.qe)
+      .map(_.toLong)
+  }
+
+  private lazy val shouldUpdatePlan: Boolean = {
+    // If the `QueryExecution` does not match the current execution ID, it means the execution ID
+    // belongs to another (parent) query, and we should call update metrics instead of plan in
+    // this query.

Review Comment:
   A cached plan in AdaptiveSparkPlan is not a subquery but we can not update plan which would overwrite the whole query plan.
   
   e.g. we can not update plan for query execution 0. Instead, we should update metrics for it.
   ```
             ...
              |
     AdaptiveSparkPlanExec (query execution 0, no execution id)
              |
     InMemoryTableScanExec
              |
             ...
              |
     AdaptiveSparkPlanExec (query execution 1, execution id 0)
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org