[jira] [Created] (HADOOP-11867) FS API: Add a high-performance vectored Read to FSDataInputStream API

["Gopal V (JIRA)" <ji...@apache.org>]

Gopal V created HADOOP-11867:
--------------------------------

             Summary: FS API: Add a high-performance vectored Read to FSDataInputStream API
                 Key: HADOOP-11867
                 URL: https://issues.apache.org/jira/browse/HADOOP-11867
             Project: Hadoop Common
          Issue Type: New Feature
    Affects Versions: 2.8.0
            Reporter: Gopal V


The most significant way to read from a filesystem in an efficient way is to let the FileSystem implementation handle the seek behaviour underneath the API to be the most efficient as possible.

A better approach to the seek problem is to provide a sequence of read locations as part of a single call, while letting the system schedule/plan the reads ahead of time.

This is exceedingly useful for seek-heavy readers on HDFS, since this allows for potentially optimizing away the seek-gaps within the FSDataInputStream implementation.

For seek+read systems with even more latency than locally-attached disks, something like a {{readFully(long[] offsets, ByteBuffer[] chunks)}} would take of the seeks internally while reading chunk.remaining() bytes into each chunk (which may be {{slice()}}ed off a bigger buffer).

The base implementation can stub in this as a sequence of seeks + read() into ByteBuffers, without forcing each FS implementation to override this in any way.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)