[GitHub] [iceberg] marton-bod commented on issue #2317: Corrupted Metadata when Catalog Fails During Commit

[GitBox <gi...@apache.org>]

marton-bod commented on issue #2317:
URL: https://github.com/apache/iceberg/issues/2317#issuecomment-796636597


   When running into a Type 2 error,  I think our retry logic would need to be changed. When retrying for a Type 2, we shouldn't cleanup the files prematurely, but instead should first attempt to reconnect to the catalog to double-check if the earlier operation succeeded. 
   
   - If we get an answer, and our snapshot is in the history of the table, then we're essentially done and no need to do anything.
   - If we get an answer, and our snapshot is not in the table history, we proceed with the file cleanup and the retry. 
   - If we cannot get ahold of the catalog persistently, then we'll give up the retry operation but still don't do any file cleanup. That ensures that if the operation did succeed, then we're not messing things up. If it was unsuccessful, that could leave some dangling files temporarily but that should be cleaned up by the Cleaner eventually, IIUC.
   
   What do you think?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org