[GitHub] [iceberg] HeartSaVioR commented on pull request #1244: Allow ExpireSnapshots to run without Deleting Files

[GitBox <gi...@apache.org>]

HeartSaVioR commented on pull request #1244:
URL: https://github.com/apache/iceberg/pull/1244#issuecomment-664811179


   I find this useful (as it is, even without the possibility of extension), as I realized expiring snapshots take very short period on actual expiration of snapshots (with committing), and bunch of time is taken from removing stale data and manifest files (I'm on 0.9.0 so still single threaded, but even with the recent patch it's limited on single process) and it looks to be shown as stuck, though it is actually doing something.
   
   If I understand correctly, as once the snapshots are expired, stale data and manifest can be also removed via RemoveOrphanFilesAction with Spark computation power and resource. It's read-only in point of table's view so no contention on optimistic locking.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org