[I] [Bug] remove_orphan_files may accidentally delete data in multitasking concurrent scenarios [paimon]

["HuangZheng423 (via GitHub)" <gi...@apache.org>]

HuangZheng423 opened a new issue, #7710:
URL: https://github.com/apache/paimon/issues/7710

   ### Search before asking
   
   - [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.
   
   
   ### Paimon version
   
   version 1.2
   
   ### Compute Engine
   
   Spark 3.5.6
   
   ### Minimal reproduce step
   
   
   table max retained snapshot is 1
   'snapshot.num-retained.max' = '1'
   
   
   2026-04-25 04:37:12,116, Task 1 completed sys.compact operation and submitted snapshot 4959
   2026-04-25 04:37:13,195, Task 1 completed snapshot expiration. Mark snapshot 4958 as expired and delete it
   2026-04-25 04:37:37,605, Task 2 completed writing and submitted snapshot 4960
   2026-04-25 04:37:38,185, Task 2 completed snapshot expiration. Mark snapshot 4959 as expired and delete it
   At the same time, Task 1 started executing the remove_orphan_files, but the current snapshot 4959 was not read because 4959 had already been deleted by Task 2, resulting in usedFiles being empty, which in turn caused deleted=candidates, meaning that all files written one day ago were marked as files to be deleted and ultimately deleted.
   
   ### What doesn't meet your expectations?
   
   Incorrect deletion of historical data
   
   ### Anything else?
   
   _No response_
   
   ### Are you willing to submit a PR?
   
   - [ ] I'm willing to submit a PR!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Bug] remove_orphan_files may accidentally delete data in multitasking concurrent scenarios [paimon]

["heye1005 (via GitHub)" <gi...@apache.org>]

heye1005 commented on issue #7710:
URL: https://github.com/apache/paimon/issues/7710#issuecomment-4326892727

   I can work on this. The issue is that LocalOrphanFilesClean.clean() doesn't check if usedFiles is empty before deleting — when concurrent snapshot expiration removes snapshots during cleanup, all files get deleted. Will submit a PR with a safety check.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org