[GitHub] [iceberg] steveloughran commented on a change in pull request #2159: AWS: Fail writes fast when S3OutputStream encountered error in async …

[GitBox <gi...@apache.org>]

steveloughran commented on a change in pull request #2159:
URL: https://github.com/apache/iceberg/pull/2159#discussion_r566213519



##########
File path: aws/src/test/java/org/apache/iceberg/aws/s3/S3OutputStreamTest.java
##########
@@ -133,6 +134,53 @@ public void testAbortMultipart() {
     }
   }
 
+  @Test(expected = IllegalStateException.class)
+  public void testFailWriteIntAfterAsyncError() throws Exception {
+    doThrow(new RuntimeException()).when(s3mock).uploadPart((UploadPartRequest) any(), (RequestBody) any());
+    S3OutputStream stream = new S3OutputStream(s3mock, randomURI(), properties);
+    stream.write(randomData(10 * 1024 * 1024));
+    final long startTs = System.currentTimeMillis();
+    while (stream.getAsyncError() == null && System.currentTimeMillis() - startTs <= 1000) {
+      Thread.sleep(1);
+    }
+    Assert.assertNotNull(stream.getAsyncError());
+    try {
+      stream.write(1);
+    } finally {
+      stream.close();
+    }
+  }
+
+  @Test(expected = IllegalStateException.class)
+  public void testFailWriteArrayAfterAsyncError() throws Exception {
+    doThrow(new RuntimeException()).when(s3mock).uploadPart((UploadPartRequest) any(), (RequestBody) any());
+    S3OutputStream stream = new S3OutputStream(s3mock, randomURI(), properties);
+    stream.write(randomData(10 * 1024 * 1024));
+    final long startTs = System.currentTimeMillis();
+    while (stream.getAsyncError() == null && System.currentTimeMillis() - startTs <= 1000) {
+      Thread.sleep(1);
+    }
+    Assert.assertNotNull(stream.getAsyncError());
+    try {
+      stream.write(new byte[16]);
+    } finally {
+      stream.close();
+    }
+  }
+
+  @Test(expected = IllegalStateException.class)
+  public void testFailCloseAfterAsyncError() throws Exception {
+    doThrow(new RuntimeException()).when(s3mock).uploadPart((UploadPartRequest) any(), (RequestBody) any());
+    S3OutputStream stream = new S3OutputStream(s3mock, randomURI(), properties);
+    stream.write(randomData(10 * 1024 * 1024));
+    final long startTs = System.currentTimeMillis();
+    while (stream.getAsyncError() == null && System.currentTimeMillis() - startTs <= 1000) {
+      Thread.sleep(1);
+    }
+    Assert.assertNotNull(stream.getAsyncError());
+    stream.close();

Review comment:
       if you were being really strict you'd want to verify th multipart complete operation was never called




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org