[GitHub] xuanyuanking commented on a change in pull request #23371: [SPARK-26223][SQL] Track metastore operation time in scan node

[GitBox <gi...@apache.org>]

xuanyuanking commented on a change in pull request #23371: [SPARK-26223][SQL] Track metastore operation time in scan node
URL: https://github.com/apache/spark/pull/23371#discussion_r243725187
 
 

 ##########
 File path: sql/core/src/main/scala/org/apache/spark/sql/execution/ui/SQLAppStatusListener.scala
 ##########
 @@ -273,6 +281,12 @@ class SQLAppStatusListener(
       exec.metricsValues = aggregateMetrics(exec)
       exec.completionTime = Some(new Date(time))
       exec.endEvents += 1
+      // Update both physicalPlanDescription and planGraph while the description changed.
 
 Review comment:
   All the changes in `SQLAppStatusListener` and `SQLExecution` in e29ad45 is to support metastore operation phase happened in `HiveTableScanExec.rawPartitions`. Different from other phases, `rawPartitions` was initialized in execution but the UI and plan description generated before execution. If we think it's not worth to change the execution end event, the changes and metrics for rawPartitions will revert together.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org