[GitHub] [iceberg] Parth-Brahmbhatt commented on a change in pull request #2328: (#2317) Stop removal of files when catalog state is uncertain - HiveCatalog

[GitBox <gi...@apache.org>]

Parth-Brahmbhatt commented on a change in pull request #2328:
URL: https://github.com/apache/iceberg/pull/2328#discussion_r595473531



##########
File path: core/src/main/java/org/apache/iceberg/SnapshotProducer.java
##########
@@ -293,8 +293,15 @@ public void commit() {
             taskOps.commit(base, updated.withUUID());
           });
 
+    } catch (CommitFailedException commitFailedException) {
+      // We have an acknowledged failure from the Catalog. We are confident that the commit has not been applied
+      Exceptions.suppressAndThrow(commitFailedException, this::cleanAll);
     } catch (RuntimeException e) {
-      Exceptions.suppressAndThrow(e, this::cleanAll);
+      LOG.error("Cannot determine whether the commit was successful or not, the underlying data files may or " +

Review comment:
       Any reason we can't make this configurable so the client is forced to indicate if they are idempotent or not explicitly?
   
   For non idempotent clients we should keep the current behavior with limited and configurable retries to check if a snapshot was committed before issuing final snapshot deletion. If the client indicates they are idempotent we can avoid deleting and throw the exception as proposed in this PR. 




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org