[GitHub] [iceberg] chenjunjiedada opened a new issue #3744: How to reduce rewrite manifest conflict after rewrite action?

[GitBox <gi...@apache.org>]

chenjunjiedada opened a new issue #3744:
URL: https://github.com/apache/iceberg/issues/3744


   Rewrite action can produce manifest with zero added files and some deleted files like following,
   
   ```
   manifest_length: 647703 partition_spec_id: 0    added_snapshot_id: 1032105285295781129  added_data_files_count: 0   existing_data_files_count: 0    deleted_data_files_count: 900   partitions: [{"contains_null": false, "contains_nan": false, "lower_bound": {"bytes": "2021121300"}, "upper_bound": {"bytes": "2021121301"}}]   added_rows_count: 0 existing_rows_count: 0  deleted_rows_count: 4799700 
   ```
   
   MergingSnapshotProducer will drop manifests without any added data files.  So when performing rewrite manifest action in parallel with data committing,  the rewrite manifest action may fail due to missing the manifest file.  We can retry the rewrite manifest action, but can we ignore validation on such kind of manifest?
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org