[jira] [Updated] (SPARK-30368) Add computed rows metric to InMemoryRelation and show in SQL UI

["Max Thompson (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-30368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Max Thompson updated SPARK-30368:
---------------------------------
    Attachment: w-metric.png

> Add computed rows metric to InMemoryRelation and show in SQL UI
> ---------------------------------------------------------------
>
>                 Key: SPARK-30368
>                 URL: https://issues.apache.org/jira/browse/SPARK-30368
>             Project: Spark
>          Issue Type: Improvement
>          Components: SQL, Web UI
>    Affects Versions: 3.0.0
>            Reporter: Max Thompson
>            Priority: Minor
>         Attachments: w-metric.png
>
>
> This is a follow-up JIRA for: https://issues.apache.org/jira/browse/SPARK-30367
> We should add a "number of computed rows" metric to InMemoryRelation. This will show the user how many rows were computed using the InMemoryRelation's cached plan (e.g. possibly zero rows if no data had to be computed, the same amount as total rows read if all rows had to be computed, some subset of the total rows read if some partitions had to be recomputed, etc) which would help with determining how much work was done for this part of the query.
> An example with the metric where the InMemoryRelation's data was fully computed from its plan:
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org