[GitHub] [iceberg] aokolnychyi commented on a change in pull request #2865: Core: Add validation for row-level deletes with rewrites

[GitBox <gi...@apache.org>]

aokolnychyi commented on a change in pull request #2865:
URL: https://github.com/apache/iceberg/pull/2865#discussion_r677009679



##########
File path: core/src/main/java/org/apache/iceberg/MergingSnapshotProducer.java
##########
@@ -62,6 +63,9 @@
       ImmutableSet.of(DataOperations.OVERWRITE, DataOperations.REPLACE, DataOperations.DELETE);
   private static final Set<String> VALIDATE_DATA_FILES_EXIST_SKIP_DELETE_OPERATIONS =
       ImmutableSet.of(DataOperations.OVERWRITE, DataOperations.REPLACE);
+  // delete files are only added in "overwrite" operations
+  private static final Set<String> VALIDATE_REPLACED_DATA_FILES_OPERATIONS =
+      ImmutableSet.of(DataOperations.OVERWRITE);

Review comment:
       I wonder whether we should introduce a new `DataOperation` for row deltas before adopting v2. Right now, we use `OVERWRITE` for deltas as well as other operations such copy-on-write MERGE and replace partitions. This means the new validation logic will apply to operations that cannot produce delete files.
   
   It probably does not matter much in this particular use case as the delete index will be empty but it is something we should do now or never.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org