You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by vi...@apache.org on 2021/07/21 03:23:11 UTC
[spark] branch master updated (305d563 -> 2653201)
This is an automated email from the ASF dual-hosted git repository.
viirya pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git.
from 305d563 [SPARK-36153][SQL][DOCS] Update transform doc to match the current code
add 2653201 [SPARK-36030][SQL] Support DS v2 metrics at writing path
No new revisions were added by this update.
Summary of changes:
.../org/apache/spark/sql/connector/read/Scan.java | 3 +-
.../spark/sql/connector/write/DataWriter.java | 9 ++
.../apache/spark/sql/connector/write/Write.java | 9 ++
.../sql/connector/catalog/InMemoryTable.scala | 22 +++++
.../datasources/FileFormatDataWriter.scala | 51 ++++++++---
.../datasources/v2/DataSourceV2Strategy.scala | 8 +-
.../datasources/v2/WriteToDataSourceV2Exec.scala | 36 +++++++-
.../spark/sql/execution/metric/CustomMetrics.scala | 5 +-
.../execution/streaming/MicroBatchExecution.scala | 4 +-
.../sql/execution/streaming/StreamExecution.scala | 6 +-
.../streaming/continuous/ContinuousExecution.scala | 6 +-
.../streaming/continuous/ContinuousWriteRDD.scala | 11 ++-
.../continuous/WriteToContinuousDataSource.scala | 4 +-
.../WriteToContinuousDataSourceExec.scala | 11 ++-
.../sources/WriteToMicroBatchDataSource.scala | 6 +-
.../sql/connector/SimpleWritableDataSource.scala | 8 +-
.../FileFormatDataWriterMetricSuite.scala | 96 +++++++++++++++++++
.../datasources/InMemoryTableMetricSuite.scala | 96 +++++++++++++++++++
.../sql/execution/metric/CustomMetricsSuite.scala | 10 +-
.../execution/ui/SQLAppStatusListenerSuite.scala | 102 ++++++++++++++++++++-
20 files changed, 456 insertions(+), 47 deletions(-)
create mode 100644 sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/FileFormatDataWriterMetricSuite.scala
create mode 100644 sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/InMemoryTableMetricSuite.scala
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org