[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #6651: Spark 3.3 write to branch snapshot

["amogh-jahagirdar (via GitHub)" <gi...@apache.org>]

amogh-jahagirdar commented on code in PR #6651:
URL: https://github.com/apache/iceberg/pull/6651#discussion_r1084484101


##########
spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/source/SparkWrite.java:
##########
@@ -290,7 +293,12 @@ public String toString() {
   private class BatchAppend extends BaseBatchWrite {
     @Override
     public void commit(WriterCommitMessage[] messages) {
-      AppendFiles append = table.newAppend();
+      AppendFiles append;
+      if (branch != null) {
+        append = table.newAppend().toBranch(branch);
+      } else {
+        append = table.newAppend();
+      }

Review Comment:
   If we do what I mentioned above we don't need if/else here. A branch will always be defined. 



##########
spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/source/SparkWrite.java:
##########
@@ -312,7 +320,13 @@ public void commit(WriterCommitMessage[] messages) {
         return;
       }
 
-      ReplacePartitions dynamicOverwrite = table.newReplacePartitions();
+      ReplacePartitions dynamicOverwrite;
+      if (branch != null) {
+        dynamicOverwrite = table.newReplacePartitions().toBranch(branch);
+      } else {
+        dynamicOverwrite = table.newReplacePartitions();
+      }

Review Comment:
   Same as above we can remove the if/else after the change for a branch to always be defined.



##########
spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/SparkWriteConf.java:
##########
@@ -304,4 +304,9 @@ public boolean caseSensitive() {
         .defaultValue(SQLConf.CASE_SENSITIVE().defaultValueString())
         .parse();
   }
+
+  public String branch() {
+    return confParser.stringConf().option(SparkWriteOptions.BRANCH).parseOptional();

Review Comment:
   Can this return main if no branch is passed in? 



##########
spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/source/SparkWrite.java:
##########
@@ -411,7 +429,12 @@ private Expression conflictDetectionFilter() {
 
     @Override
     public void commit(WriterCommitMessage[] messages) {
-      OverwriteFiles overwriteFiles = table.newOverwrite();
+      OverwriteFiles overwriteFiles;
+      if (branch != null) {
+        overwriteFiles = table.newOverwrite().toBranch(branch);
+      } else {
+        overwriteFiles = table.newOverwrite();
+      }

Review Comment:
   Same as above



##########
spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/source/SparkWrite.java:
##########
@@ -349,8 +363,12 @@ private OverwriteByFilter(Expression overwriteExpr) {
 
     @Override
     public void commit(WriterCommitMessage[] messages) {
-      OverwriteFiles overwriteFiles = table.newOverwrite();
-      overwriteFiles.overwriteByRowFilter(overwriteExpr);
+      OverwriteFiles overwriteFiles;
+      if (branch != null) {
+        overwriteFiles = table.newOverwrite().toBranch(branch).overwriteByRowFilter(overwriteExpr);
+      } else {
+        overwriteFiles = table.newOverwrite().overwriteByRowFilter(overwriteExpr);

Review Comment:
   Same as above we can remove the if/else since a branch will always be defined.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org