(iceberg) branch main updated: Spark 3.5: Set useCommitCoordinator to false in streaming writes (#9027)

[ao...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

aokolnychyi pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/iceberg.git


The following commit(s) were added to refs/heads/main by this push:
     new 13fd06d90d Spark 3.5: Set useCommitCoordinator to false in streaming writes (#9027)
13fd06d90d is described below

commit 13fd06d90ddc6f5c6bacf79b2c0dbe64838e7b65
Author: Huaxin Gao <hu...@apple.com>
AuthorDate: Sat Nov 11 10:15:38 2023 -0800

    Spark 3.5: Set useCommitCoordinator to false in streaming writes (#9027)
    
    Co-authored-by: Huaxin Gao <hu...@apple.com>
---
 .../src/main/java/org/apache/iceberg/spark/source/SparkWrite.java    | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/SparkWrite.java b/spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/SparkWrite.java
index a7c1234a79..58d77695f2 100644
--- a/spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/SparkWrite.java
+++ b/spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/SparkWrite.java
@@ -506,6 +506,11 @@ abstract class SparkWrite implements Write, RequiresDistributionAndOrdering {
       return createWriterFactory();
     }
 
+    @Override
+    public boolean useCommitCoordinator() {
+      return false;
+    }
+
     @Override
     public final void commit(long epochId, WriterCommitMessage[] messages) {
       LOG.info("Committing epoch {} for query {} in {} mode", epochId, queryId, mode());