[jira] [Created] (SPARK-38322) Support query stage show runtime statistics in formatted explain mode

["XiDuo You (Jira)" <ji...@apache.org>]

XiDuo You created SPARK-38322:
---------------------------------

             Summary: Support query stage show runtime statistics in formatted explain mode
                 Key: SPARK-38322
                 URL: https://issues.apache.org/jira/browse/SPARK-38322
             Project: Spark
          Issue Type: Improvement
          Components: SQL
    Affects Versions: 3.3.0
            Reporter: XiDuo You


The formatted explalin mode is the powerful explain mode to show the details of query plan. In AQE, the query stage know its statistics if has already materialized. So it can help to quick check the conversion of plan, e.g. join selection. 

 

A simple example:
{code:java}
SELECT * FROM t JOIN t2 ON t.c = t2.c;{code}
 

 
{code:java}
== Physical Plan ==
AdaptiveSparkPlan (21)
+- == Final Plan ==
   * SortMergeJoin Inner (13)
   :- * Sort (6)
   :  +- AQEShuffleRead (5)
   :     +- ShuffleQueryStage (4), Statistics(sizeInBytes=16.0 B, rowCount=1)
   :        +- Exchange (3)
   :           +- * Filter (2)
   :              +- Scan hive default.t (1)
   +- * Sort (12)
      +- AQEShuffleRead (11)
         +- ShuffleQueryStage (10), Statistics(sizeInBytes=16.0 B, rowCount=1)
            +- Exchange (9)
               +- * Filter (8)
                  +- Scan hive default.t2 (7)
+- == Initial Plan ==
   SortMergeJoin Inner (20)
   :- Sort (16)
   :  +- Exchange (15)
   :     +- Filter (14)
   :        +- Scan hive default.t (1)
   +- Sort (19)
      +- Exchange (18)
         +- Filter (17)
            +- Scan hive default.t2 (7){code}
 

 



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org