[GitHub] [iceberg] chenjunjiedada commented on a diff in pull request #4703: API: Optionally ignore position deletes in rewrite validation

[GitBox <gi...@apache.org>]

chenjunjiedada commented on code in PR #4703:
URL: https://github.com/apache/iceberg/pull/4703#discussion_r911521950


##########
api/src/main/java/org/apache/iceberg/RewriteFiles.java:
##########
@@ -84,4 +84,12 @@ RewriteFiles rewriteFiles(Set<DataFile> dataFilesToReplace, Set<DeleteFile> dele
    * @return this for method chaining
    */
   RewriteFiles validateFromSnapshot(long snapshotId);
+
+  /**
+   * Ignore the position deletes in rewrite validation. Flink upsert job only generates position deletes in the
+   * ongoing transaction, so it is not necessary to validate position deletes when rewriting.
+   *
+   * @return this for method chaining
+   */
+  RewriteFiles ignorePosDeletesInValidation();

Review Comment:
   Plus, we only support writer bloom filter on parquet format right now so it cannot support other format delete? Another way is to write bloom filters of deletes on puffin, not sure if that could work.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org