[GitHub] [iceberg] RussellSpitzer commented on a diff in pull request #7637: Spark 3.4: Distribution and ordering enhancements

["RussellSpitzer (via GitHub)" <gi...@apache.org>]

RussellSpitzer commented on code in PR #7637:
URL: https://github.com/apache/iceberg/pull/7637#discussion_r1200818149


##########
spark/v3.4/spark/src/main/java/org/apache/iceberg/spark/SparkWriteConf.java:
##########
@@ -262,7 +278,50 @@ private DistributionMode defaultWriteDistributionMode() {
     }
   }
 
-  public DistributionMode deleteDistributionMode() {
+  public SparkWriteRequirements copyOnWriteRequirements(Command command) {
+    if (ignoreTableDistributionAndOrdering()) {
+      LOG.info("Skipping distribution/ordering: disabled per job configuration");
+      return SparkWriteRequirements.EMPTY;
+    }
+
+    return SparkWriteUtil.copyOnWriteRequirements(
+        table, command, copyOnWriteDistributionMode(command), fanoutWriterEnabled());
+  }
+
+  @VisibleForTesting
+  DistributionMode copyOnWriteDistributionMode(Command command) {
+    switch (command) {
+      case DELETE:
+        return deleteDistributionMode();
+      case UPDATE:
+        return updateDistributionMode();
+      case MERGE:
+        return copyOnWriteMergeDistributionMode();
+      default:
+        throw new IllegalArgumentException("Unexpected command: " + command);
+    }
+  }
+
+  public SparkWriteRequirements positionDeltaRequirements(Command command) {
+    return SparkWriteUtil.positionDeltaRequirements(

Review Comment:
   Let's make sure to follow up on this in the spec. The sorting requirement I think is a little odd, but if we switch to another layout, like a delete bitmap per file this won't really matter.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org