[GitHub] [hadoop] steveloughran commented on a diff in pull request #4294: HADOOP-18221. Drains stream async before closing

[GitBox <gi...@apache.org>]

steveloughran commented on code in PR #4294:
URL: https://github.com/apache/hadoop/pull/4294#discussion_r875826424


##########
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/read/S3File.java:
##########
@@ -214,7 +223,40 @@ void close(InputStream inputStream) {
       this.s3Objects.remove(inputStream);
     }
 
-    Io.closeIgnoringIoException(inputStream);
-    Io.closeIgnoringIoException(obj);
+    this.futurePool.executeRunnable(new DrainTask(inputStream, obj));
   }
+
+
+  /**
+   * Drain task that is submitted to the future pool.
+   */
+  private static class DrainTask implements Runnable {
+
+    private final InputStream inputStream;
+    private final S3Object obj;
+    private long drained;
+
+    DrainTask(InputStream inputStream, S3Object obj) {
+      this.inputStream = inputStream;
+      this.obj = obj;
+    }
+
+    @Override
+    public void run() {
+      try {
+
+        while(this.inputStream.read() >= 0) {

Review Comment:
   look at the changes in hadoop trunk s3a input stream here...it reads into a buffer for draining, and is marginally faster



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org