[spark] branch branch-3.0 updated: [SPARK-30563][SQL] Disable using commit coordinator with NoopDataSource

[gu...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

gurwls223 pushed a commit to branch branch-3.0
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/branch-3.0 by this push:
     new 6a7aa0d  [SPARK-30563][SQL] Disable using commit coordinator with NoopDataSource
6a7aa0d is described below

commit 6a7aa0d1666aa57213ef16cc858dbf105f0810f4
Author: Peter Toth <pe...@gmail.com>
AuthorDate: Fri Mar 6 10:30:59 2020 +0900

    [SPARK-30563][SQL] Disable using commit coordinator with NoopDataSource
    
    ### What changes were proposed in this pull request?
    This PR disables using commit coordinator with `NoopDataSource`.
    
    ### Why are the changes needed?
    No need for a coordinator in benchmarks.
    
    ### Does this PR introduce any user-facing change?
    No.
    
    ### How was this patch tested?
    Existing UTs.
    
    Closes #27791 from peter-toth/SPARK-30563-disalbe-commit-coordinator.
    
    Authored-by: Peter Toth <pe...@gmail.com>
    Signed-off-by: HyukjinKwon <gu...@apache.org>
    (cherry picked from commit 72b52a3cdf2b8661c0f82db645e805ff6b085dfc)
    Signed-off-by: HyukjinKwon <gu...@apache.org>
---
 .../org/apache/spark/sql/execution/datasources/noop/NoopDataSource.scala | 1 +
 1 file changed, 1 insertion(+)

diff --git a/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/noop/NoopDataSource.scala b/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/noop/NoopDataSource.scala
index 4fad0a2..851cc51 100644
--- a/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/noop/NoopDataSource.scala
+++ b/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/noop/NoopDataSource.scala
@@ -61,6 +61,7 @@ private[noop] object NoopWriteBuilder extends WriteBuilder with SupportsTruncate
 private[noop] object NoopBatchWrite extends BatchWrite {
   override def createBatchWriterFactory(info: PhysicalWriteInfo): DataWriterFactory =
     NoopWriterFactory
+  override def useCommitCoordinator(): Boolean = false
   override def commit(messages: Array[WriterCommitMessage]): Unit = {}
   override def abort(messages: Array[WriterCommitMessage]): Unit = {}
 }


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org