[GitHub] [iceberg] RussellSpitzer commented on issue #4686: Spark: CommitFailedException in Spark Commit Operation Can Lead to Data Loss

[GitBox <gi...@apache.org>]

RussellSpitzer commented on issue #4686:
URL: https://github.com/apache/iceberg/issues/4686#issuecomment-1116721402

   Sorry incorrect exception there should have been CommitStateUnknown, the exception is not swallowed. The issue is that the operation.commit code in SnapshotProducer can throw CommitStateUnknown which will be surfaced up to Spark. This is explicitly rethrown see
   
   https://github.com/apache/iceberg/blob/674dcf1c2e3cb0444a183c55a24b521e297880de/core/src/main/java/org/apache/iceberg/SnapshotProducer.java#L321
   
   Once rethrown into the SparkWrite commitOperation code it will trigger the data source Abort method which will delete the data files.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org