[GitHub] [iceberg] zhongyujiang commented on pull request #3772: [Api][Spark]Add cleanExpiredFiles to Actions/ExpireSnapshots

[GitBox <gi...@apache.org>]

zhongyujiang commented on pull request #3772:
URL: https://github.com/apache/iceberg/pull/3772#issuecomment-997856672


   @ajantha-bhat @rdblue Thanks for your review! Maybe I didn't make it clear, the failure of _expire_snapshots_ I met is not caused by deleting files instead of building expired files dataset, cause some files needed to build expiredFiles are deleted by _remove_orphan_files_ which happens to be started right after _expire_snapshots_' commit. So plugin dummy consumer method to avoid deleting the files has no effect on this situation.
   This situation is quite rare, but truely it happens. I think a simple way to avoid this is execute  _remove_orphan_files_ after _expire_snapshots_ job is done, but our _remove_orphan_files_ procedure is automatically scheduled by program. So it's a bit difficult to control timing.
   
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org