You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@spark.apache.org by we...@apache.org on 2021/04/20 15:02:17 UTC

[spark] branch master updated (3614448 -> eb9a439)

This is an automated email from the ASF dual-hosted git repository.

wenchen pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git.


    from 3614448  [SPARK-34035][SQL] Refactor ScriptTransformation to remove input parameter and replace it by child.output
     add eb9a439  [SPARK-34338][SQL] Report metrics from Datasource v2 scan

No new revisions were added by this update.

Summary of changes:
 .../spark/sql/connector/read/PartitionReader.java  |  3 +-
 .../execution/datasources/v2/BatchScanExec.scala   |  2 +-
 .../datasources/v2/ContinuousScanExec.scala        |  3 +-
 .../execution/datasources/v2/DataSourceRDD.scala   | 20 ++++-
 .../datasources/v2/DataSourceV2ScanExecBase.scala  | 10 ++-
 .../datasources/v2/MicroBatchScanExec.scala        |  2 +-
 .../spark/sql/execution/metric/CustomMetrics.scala | 79 ++++++++++++++++++++
 .../spark/sql/execution/metric/SQLMetrics.scala    | 10 +++
 .../continuous/ContinuousDataSourceRDD.scala       |  8 +-
 .../continuous/ContinuousQueuedDataReader.scala    |  3 +
 .../sql/execution/ui/SQLAppStatusListener.scala    | 43 +++++++++--
 .../sql/execution/metric/CustomMetricsSuite.scala  | 54 ++++++++++++++
 .../execution/ui/SQLAppStatusListenerSuite.scala   | 87 ++++++++++++++++++++++
 13 files changed, 308 insertions(+), 16 deletions(-)
 create mode 100644 sql/core/src/main/scala/org/apache/spark/sql/execution/metric/CustomMetrics.scala
 create mode 100644 sql/core/src/test/scala/org/apache/spark/sql/execution/metric/CustomMetricsSuite.scala

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org