[GitHub] [iceberg] jackye1995 commented on a diff in pull request #7262: AWS: abort S3 input stream on close if not EOS

["jackye1995 (via GitHub)" <gi...@apache.org>]

jackye1995 commented on code in PR #7262:
URL: https://github.com/apache/iceberg/pull/7262#discussion_r1156352749


##########
aws/src/main/java/org/apache/iceberg/aws/s3/S3InputStream.java:
##########
@@ -194,9 +195,29 @@ private void openStream() throws IOException {
     }
   }
 
-  private void closeStream() throws IOException {
+  private void closeStream() {
     if (stream != null) {
-      stream.close();
+      // if we aren't at the end of the stream, and the stream is abortable, then
+      // call abort() so we don't read the remaining data with the Apache HTTP client
+      abortStream();
+      try {
+        stream.close();
+      } catch (Exception e) {
+        // log at trace level as closing an aborted stream will throw a content length
+        // check exception with the Apache HTTP client
+        LOG.trace("Error closing stream", e);
+      }
+      stream = null;
+    }
+  }
+
+  private void abortStream() {
+    try {
+      if (stream instanceof Abortable && stream.read() != -1) {

Review Comment:
   why do we want to read one more byte here? It might cause one more request. I think it does not hurt to even abort when it's already fully read.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org