[GitHub] [iceberg] abmo-x commented on a diff in pull request #6818: Fix: add_files data loss - old partitions get lost when new partition is added

["abmo-x (via GitHub)" <gi...@apache.org>]

abmo-x commented on code in PR #6818:
URL: https://github.com/apache/iceberg/pull/6818#discussion_r1105115220


##########
spark/v3.3/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestAddFilesProcedure.java:
##########
@@ -394,6 +397,29 @@ public void addPartitionToPartitioned() {
         sql("SELECT id, name, dept, subdept FROM %s ORDER BY id", tableName));
   }
 
+  @Test
+  public void addPartitionToPartitionedSnapshotIdInheritanceEnabled() {
+    createPartitionedFileTable("parquet");
+
+    String createIceberg =
+        "CREATE TABLE %s (id Integer, name String, dept String, subdept String) USING iceberg PARTITIONED BY (id)"
+            + "TBLPROPERTIES ('%s'='true')";
+
+    sql(createIceberg, tableName, TableProperties.SNAPSHOT_ID_INHERITANCE_ENABLED);

Review Comment:
   yes, this happens only when its true, when the if condition is `false` the `copyManifest` uses [`newManifestOutput()` ](https://github.com/apache/iceberg/blob/ef7e20e12eae2638015ff91b3356eb2f32f601cc/core/src/main/java/org/apache/iceberg/MergingSnapshotProducer.java#L260) which appends commitUUID to the file name making it unique.
   
   ```
   protected void add(ManifestFile manifest) {
       Preconditions.checkArgument(
           manifest.content() == ManifestContent.DATA, "Cannot append delete manifest: %s", manifest);
       if (snapshotIdInheritanceEnabled && manifest.snapshotId() == null) {
         appendedManifestsSummary.addedManifest(manifest);
         appendManifests.add(manifest);
       } else {
         // the manifest must be rewritten with this update's snapshot ID
         ManifestFile copiedManifest = copyManifest(manifest);
         rewrittenAppendManifests.add(copiedManifest);
       }
     }
   ```
   https://github.com/apache/iceberg/blob/ef7e20e12eae2638015ff91b3356eb2f32f601cc/core/src/main/java/org/apache/iceberg/MergingSnapshotProducer.java#L244



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org