You are viewing a plain text version of this content. The canonical link for it is here.

Posted to hdfs-dev@hadoop.apache.org by "Abdullah Alamoudi (JIRA)" <ji...@apache.org> on 2014/06/30 11:19:24 UTC

[jira] [Created] (HDFS-6607) DFSInputStream Seek performance improvement

Abdullah Alamoudi created HDFS-6607:
---------------------------------------

             Summary: DFSInputStream Seek performance improvement
                 Key: HDFS-6607
                 URL: https://issues.apache.org/jira/browse/HDFS-6607
             Project: Hadoop HDFS
          Issue Type: Improvement
          Components: hdfs-client, performance
    Affects Versions: 2.4.1
            Reporter: Abdullah Alamoudi
            Priority: Minor


When having a DFSInputStream open and seeking to a position that resides in the same block, if the target position is in the TCP buffer already, the seek is performed efficiently simply by eating up the intervening data. See line 1368 in the file: hadoop-common/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java.

However, if the position is in the same block but after the TCP buffer, the inputstream performs a set of actions including closing the current block reader, locating the block again, selecting a data node and creating a new block reader. During this, many objects are created and all of this is very inefficient for users with random access needs (e.g index access).



--
This message was sent by Atlassian JIRA
(v6.2#6252)