[jira] [Created] (SPARK-30368) Add computed rows metric to InMemoryRelation and show in SQL UI

["Max Thompson (Jira)" <ji...@apache.org>]

Max Thompson created SPARK-30368:
------------------------------------

             Summary: Add computed rows metric to InMemoryRelation and show in SQL UI
                 Key: SPARK-30368
                 URL: https://issues.apache.org/jira/browse/SPARK-30368
             Project: Spark
          Issue Type: Improvement
          Components: SQL, Web UI
    Affects Versions: 3.0.0
            Reporter: Max Thompson
         Attachments: w-metric.png

This is a follow-up JIRA for: https://issues.apache.org/jira/browse/SPARK-30367

We should add a "number of computed rows" metric to InMemoryRelation. This will show the user how many rows were computed using the InMemoryRelation's cached plan (e.g. possibly zero rows if no data had to be computed, the same amount as total rows read if all rows had to be computed, some subset of the total rows read if some partitions had to be recomputed, etc) which would help with determining how much work was done for this part of the query.

An example with the metric where the InMemoryRelation's data was fully computed from its plan:

 

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org