[jira] [Commented] (HDFS-13513) BenchmarkThroughput.readFile hangs with misconfigured BUFFER_SIZE

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HDFS-13513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16459416#comment-16459416 ] 

ASF GitHub Bot commented on HDFS-13513:
---------------------------------------

Github user functicons commented on a diff in the pull request:

    https://github.com/apache/hadoop/pull/375#discussion_r185160425
  
    --- Diff: hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/BenchmarkThroughput.java ---
    @@ -134,9 +134,9 @@ private void readFile(FileSystem fs,
         InputStream in = fs.open(f);
         byte[] data = new byte[BUFFER_SIZE];
         long val = 0;
    -    while (val >= 0) {
    +    do {
           val = in.read(data);
    -    }
    +    } while (val > 0);
    --- End diff --
    
    I don't think this is fix is correct. According to the [InputStream.read](762db5bde1c69b54458bea8eb69aa544636c709a) doc, "the total number of bytes read into the buffer, or -1 if there is no more data because the end of the stream has been reached". It is possible that BUFFER_SIZE > 0 and val == 0, in this case, it should continue to read, otherwise it will miss the data. Instead, I think we should check BUFFER_SIZE explicitly.


> BenchmarkThroughput.readFile hangs with misconfigured BUFFER_SIZE
> -----------------------------------------------------------------
>
>                 Key: HDFS-13513
>                 URL: https://issues.apache.org/jira/browse/HDFS-13513
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: test
>            Reporter: John Doe
>            Priority: Minor
>             Fix For: 2.5.0
>
>
> When the BUFFER_SIZE is configured to be 0, the while loop in BenchmarkThroughput.readFile function hangs endlessly.
> This is because when the data.size (i.e., BUFFER_SIZE) is 0, the val will always be 0 by invoking val=in.read(data).
> Here is the code snippet.
> {code:java}
>   BUFFER_SIZE = conf.getInt("dfsthroughput.buffer.size", 4 * 1024);//when dfsthroughput.buffer.size is configued to be 0
>   private void readFile(FileSystem fs, Path f, String name, Configuration conf ) throws IOException {
>     System.out.print("Reading " + name);
>     resetMeasurements();
>     InputStream in = fs.open(f);
>     byte[] data = new byte[BUFFER_SIZE];
>     long val = 0;
>     while (val >= 0) {
>       val = in.read(data);
>     }
>     in.close();
>     printMeasurements();
>   }
> {code}
> The similar case is [HDFS-13514|https://issues.apache.org/jira/browse/HDFS-13514]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org