You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "JohnFitzpatrick44 (via GitHub)" <gi...@apache.org> on 2026/07/20 15:11:58 UTC
[I] Commits that remove and re-add a file can delete a live file when expiring snapshots [iceberg]
JohnFitzpatrick44 opened a new issue, #17304:
URL: https://github.com/apache/iceberg/issues/17304
### Apache Iceberg version
1.11.0 (latest release)
### Query engine
None
### Please describe the bug 🐞
If a data file is removed and re-added in the same commit, e.g. during a `RewriteFiles` commit where `addFile` and `deleteFile` are called on the same file path, that data file (which is still live) can be deleted when expiring snapshots, corrupting the table.
This occurs when `IncrementalFileCleanup.findFilesToDelete` (https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/IncrementalFileCleanup.java#L302-L305) removes files from snapshots that are `DELETED` without examining other manifest entries. This can result in a live file being deleted.
Here's the order of events:
1. A writer does a `RewriteFiles` commit. The commit removes file `F` and adds file `F` at the same path. Snapshot `S1` contains one `DELETED` and `ADDED` entry for `F`.
2. `F` remains live, and readers continue using `F` correctly.
3. Another snapshot `S2` is created.
4. `expireSnapshots` removes `S1`. Conditions are met so that `IncrementalFileCleanup` is used.
5. The cleanup finds the `DELETED` entry for `F` and deletes it.
6. `S2` still refers to `F`, so readers now fail.
```java
// table with file F live in the current snapshot
table.newRewrite()
.rewriteFiles(Set.of(F), Set.of(F)) // remove F, add F at the same path
.commit(); // snapshot S1
table.newAppend().appendFile(G).commit(); // snapshot S2
table.expireSnapshots()
.expireOlderThan(System.currentTimeMillis())
.commit(); // deletes F from storage
// reads now fail, as F is referenced but not found
```
We can address this in the `RewriteFile` commit step or `ExpireSnapshots` step.
1. `MergingSnapshotProducer` rejects or no-ops on commits that remove and add the same file.
2. `IncrementalFileCleanup` does not delete files that are removed and added in the same step. Alternatively, `ReachableFileCleanup` is used more frequently, but this would deteriorate performance.
From https://github.com/apache/iceberg/issues/13568 / https://github.com/apache/iceberg/pull/13614, it seems that option 2 would be preferred, and/or we should deprecate `IncrementalFileCleanup`.
### Willingness to contribute
- [ ] I can contribute a fix for this bug independently
- [x] I would be willing to contribute a fix for this bug with guidance from the Iceberg community
- [ ] I cannot contribute a fix for this bug at this time
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Commits that remove and re-add a file can delete a live file when expiring snapshots [iceberg]
Posted by "RussellSpitzer (via GitHub)" <gi...@apache.org>.
RussellSpitzer commented on issue #17304:
URL: https://github.com/apache/iceberg/issues/17304#issuecomment-5072248502
"Undelete" is not my favorite thing for reasons like this. We have a lot of assumptions (and other implementations as do as well) that a file will not be added back to the table after having been removed.
Just wondering, did we hit this bug in a real world use case? I just ask since most of the implementations I know of use a reachable approach and not the incremental one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Commits that remove and re-add a file can delete a live file when expiring snapshots [iceberg]
Posted by "JohnFitzpatrick44 (via GitHub)" <gi...@apache.org>.
JohnFitzpatrick44 commented on issue #17304:
URL: https://github.com/apache/iceberg/issues/17304#issuecomment-5072648619
Yes, the trigger was an expire snapshots run that used the incremental path.
I'll close this out, as it is clearly noted in the spec. Sacrificing performance to enforce boundaries here doesn't seem like a good idea.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Commits that remove and re-add a file can delete a live file when expiring snapshots [iceberg]
Posted by "Tishj (via GitHub)" <gi...@apache.org>.
Tishj commented on issue #17304:
URL: https://github.com/apache/iceberg/issues/17304#issuecomment-5051663612
I'm sorry if this comment isn't helpful, but the spec mentions this exact thing as being problematic, saying it should be avoided:
> It is not recommended to add a deleted file back to a table. Adding a deleted file can lead to edge cases where incremental deletes can break table snapshots.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Commits that remove and re-add a file can delete a live file when expiring snapshots [iceberg]
Posted by "JohnFitzpatrick44 (via GitHub)" <gi...@apache.org>.
JohnFitzpatrick44 closed issue #17304: Commits that remove and re-add a file can delete a live file when expiring snapshots
URL: https://github.com/apache/iceberg/issues/17304
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org