[GitHub] [iceberg] openinx commented on a change in pull request #2499: Spark: Add builder pattern to SparkAppenderFactory to handle increasing number of arguments

[GitBox <gi...@apache.org>]

openinx commented on a change in pull request #2499:
URL: https://github.com/apache/iceberg/pull/2499#discussion_r619946868



##########
File path: spark/src/main/java/org/apache/iceberg/spark/source/SparkAppenderFactory.java
##########
@@ -85,6 +72,52 @@
     this.posDeleteRowSchema = posDeleteRowSchema;
   }
 
+  public static SparkAppenderFactoryBuilder builderFor(Table table, Schema writeSchema, StructType dsSchema) {
+    return new SparkAppenderFactoryBuilder(table, writeSchema, dsSchema);
+  }
+
+  public static class SparkAppenderFactoryBuilder {
+    private final Map<String, String> properties;
+    private final Schema writeSchema;
+    private final StructType dsSchema;
+    private PartitionSpec spec;
+    private int[] equalityFieldIds;
+    private Schema eqDeleteRowSchema;
+    private Schema posDeleteRowSchema;
+
+    SparkAppenderFactoryBuilder(Table table, Schema writeSchema, StructType dsSchema) {
+      this.properties = table.properties();
+      this.spec = table.spec();
+      this.writeSchema = writeSchema;
+      this.dsSchema = dsSchema;
+    }
+
+    public SparkAppenderFactoryBuilder spec(PartitionSpec newSpec) {
+      this.spec = newSpec;
+      return this;
+    }
+
+    public SparkAppenderFactoryBuilder equalityFieldIds(int[] newEqualityFieldIds) {
+      this.equalityFieldIds = newEqualityFieldIds;
+      return this;
+    }
+
+    public SparkAppenderFactoryBuilder eqDeleteRowSchema(Schema newEqDeleteRowSchema) {
+      this.eqDeleteRowSchema = newEqDeleteRowSchema;
+      return this;
+    }
+
+    public SparkAppenderFactoryBuilder posDelRowSchema(Schema newPosDelRowSchema) {
+      this.posDeleteRowSchema = newPosDelRowSchema;
+      return this;
+    }
+
+    public SparkAppenderFactory build() {
+      return new SparkAppenderFactory(properties, writeSchema, dsSchema, spec, equalityFieldIds,

Review comment:
       We will need to do the necessary validation before we construct the `SparkAppenderFactory` instance: 
   
   1.  The `properties`, `writeSchema`, `dsSchema`  must not be null;
   2.  If the `equalityFieldIds` is not null,  means people plan to write equality deletes into delete files. Then the `eqDeleteRowSchema` must not be null;  Another side,  if `eqDeleteRowSchema` is not null, then `equalityFieldIds` must not be null.
   3.  the `posDeleteRowSchema` can always choose to be null or not.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org