[jira] [Created] (HDFS-16520) Improve EC pread: avoid potential reading whole block

["daimin (Jira)" <ji...@apache.org>]

daimin created HDFS-16520:
-----------------------------

             Summary: Improve EC pread: avoid potential reading whole block
                 Key: HDFS-16520
                 URL: https://issues.apache.org/jira/browse/HDFS-16520
             Project: Hadoop HDFS
          Issue Type: Improvement
          Components: dfsclient, ec
    Affects Versions: 3.3.2, 3.3.1
            Reporter: daimin
            Assignee: daimin


HDFS client 'pread' represents 'position read', this kind of read just need a range of data instead of reading the whole file/block. By using BlockReaderFactory#setLength, client tells datanode the block length to be read from disk and sent to client.
To EC file, the block length to read is not well set, by default using 'block.getBlockSize() - offsetInBlock' to both pread and sread. Thus datanode read much more data and send to client, and abort when client closes connection. There is a lot waste of resource to this situation.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org