[GitHub] [iceberg] RussellSpitzer commented on a diff in pull request #5311: S3OutputStream - failure to close should persist on subsequent close calls

[GitBox <gi...@apache.org>]

RussellSpitzer commented on code in PR #5311:
URL: https://github.com/apache/iceberg/pull/5311#discussion_r933455961


##########
aws/src/main/java/org/apache/iceberg/aws/s3/S3OutputStream.java:
##########
@@ -240,6 +241,13 @@ private void newStream() throws IOException {
 
   @Override
   public void close() throws IOException {
+
+    // A failed s3 close removes state that is required for a successful close.
+    if (closeFailureException != null) {
+      throw new IOException(
+          "Attempted to close a S3 stream that failed to close earlier", closeFailureException);

Review Comment:
   @rdblue We had a discussion about this offline and I think this is probably the only safe thing to do. While double-closing shouldn't throw an exception, if the close failed that means that any future close is also a failure. The S3OutputStream loses it's internal state on close which means that a subsequent close call can never succeed if the IOException is handled.
   
   Because we can't ever retry successfully we decided that the stream should keep throwing exceptions to allow a caller to not falsely believe that a retry close had succeeded when the original close failed.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org