[GitHub] [iceberg] rdblue commented on a diff in pull request #4703: API: Optionally ignore position deletes in rewrite validation

[GitBox <gi...@apache.org>]

rdblue commented on code in PR #4703:
URL: https://github.com/apache/iceberg/pull/4703#discussion_r889071134


##########
api/src/main/java/org/apache/iceberg/RewriteFiles.java:
##########
@@ -84,4 +84,12 @@ RewriteFiles rewriteFiles(Set<DataFile> dataFilesToReplace, Set<DeleteFile> dele
    * @return this for method chaining
    */
   RewriteFiles validateFromSnapshot(long snapshotId);
+
+  /**
+   * Ignore the position deletes in rewrite validation. Flink upsert job only generates position deletes in the
+   * ongoing transaction, so it is not necessary to validate position deletes when rewriting.
+   *
+   * @return this for method chaining
+   */
+  RewriteFiles ignorePosDeletesInValidation();

Review Comment:
   I agree with Anton's approach. I think we can automatically detect cases where we can do this optimization. That's much safer than adding a config.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org