[GitHub] [hadoop] steveloughran commented on a change in pull request #983: HADOOP-16379: S3AInputStream#unbuffer should merge input stream stats into fs-wide stats

[GitBox <gi...@apache.org>]

steveloughran commented on a change in pull request #983: HADOOP-16379: S3AInputStream#unbuffer should merge input stream stats into fs-wide stats
URL: https://github.com/apache/hadoop/pull/983#discussion_r294865239
 
 

 ##########
 File path: hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AUnbuffer.java
 ##########
 @@ -60,6 +70,66 @@ public void testUnbuffer() throws IOException {
     }
   }
 
+  /**
+   * Test that calling {@link S3AInputStream#unbuffer()} merges a stream's
+   * {@link org.apache.hadoop.fs.s3a.S3AInstrumentation.InputStreamStatistics}
+   * into the {@link S3AFileSystem}'s {@link S3AInstrumentation} instance.
+   */
+  @Test
+  public void testUnbufferStreamStatistics() throws IOException {
+    describe("testUnbufferStreamStatistics");
+
+    // Create a new S3AFileSystem instance so we can have an independent
+    // instance of S3AInstrumentation
+    Configuration conf = createConfiguration();
+    S3AFileSystem fs = new S3AFileSystem();
+    fs.initialize(getFileSystem().getUri(), conf);
+
+    // Open file, read half the data, and then call unbuffer
+    FSDataInputStream inputStream = null;
+    try {
+      inputStream = fs.open(dest);
+
+      // Sanity check to make sure the stream statistics are 0
+      assertEquals(0,
+              fs.getInstrumentation().getCounterValue(STREAM_SEEK_BYTES_READ));
+
+      assertEquals(8, inputStream.read(new byte[8]));
+      inputStream.unbuffer();
+
+      // Validate that calling unbuffer updates the input stream statistics
+      assertEquals(8,
+              fs.getInstrumentation().getCounterValue(STREAM_SEEK_BYTES_READ));
+
+      // Validate that calling unbuffer twice in a row updates the statistics
+      // correctly
+      assertEquals(4, inputStream.read(new byte[4]));
+      inputStream.unbuffer();
+      assertEquals(12,
+              fs.getInstrumentation().getCounterValue(STREAM_SEEK_BYTES_READ));
+    } finally {
+      if (inputStream != null) {
 
 Review comment:
   use `IOUtils.closeStream`

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org