[jira] [Commented] (HDFS-1323) Pool/share file channels for HDFS read

["Todd Lipcon (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HDFS-1323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13044678#comment-13044678 ] 

Todd Lipcon commented on HDFS-1323:
-----------------------------------

Some thoughts on the patch while I was updating it:
- The ReadChannelPool class needs to have some updated javadoc explaining what exactly it does.
- The synchronization isn't entirely clear within the class - looks like some things are both using the rwlock and concurrent collections, which always makes me suspicious
- it probably makes more sense for this to be a least-recently-used cache rather than least-recently-created
- not clear why we have a high-water and low-water mark, instead of just a single limit, where we close exactly one channel if we exceed the limit.
- should be a conf to completely take the cache out of the loop

> Pool/share file channels for HDFS read
> --------------------------------------
>
>                 Key: HDFS-1323
>                 URL: https://issues.apache.org/jira/browse/HDFS-1323
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: data-node
>            Reporter: Jay Booth
>         Attachments: hdfs-1323-20100730.patch, hdfs-1323-trunk.txt
>
>
> Currently, all reads in HDFS require opening and closing the underlying block/meta filechannels.  We could pool these filechannels and save some system calls and other work.  Since HDFS read requests can be satisfied by positioned reads and transferTos, we can even share these filechannels between concurrently executing requests.
> The attached patch was benchmarked as part of work on HDFS-918 and exhibited a 10% performance increase for small random reads.
> This does not affect client logic and involves minimal change to server logic.  Patch is based on branch 20-append. 

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira