[jira] [Resolved] (HADOOP-18391) Improve VectoredReadUtils#readVectored() for direct buffers

["Mukund Thakur (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HADOOP-18391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Mukund Thakur resolved HADOOP-18391.
------------------------------------
    Resolution: Fixed

merged to branch-3.3

> Improve VectoredReadUtils#readVectored() for direct buffers
> -----------------------------------------------------------
>
>                 Key: HADOOP-18391
>                 URL: https://issues.apache.org/jira/browse/HADOOP-18391
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs
>    Affects Versions: 3.3.9
>            Reporter: Steve Loughran
>            Assignee: Mukund Thakur
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 3.4.0
>
>
> harden the VectoredReadUtils methods for consistent and more robust use, especially in those filesystems which don't have the api.
> VectoredReadUtils.readInDirectBuffer should allocate a max buffer size, .e.g 4mb, then do repeated reads and copies; this ensures that you don't OOM with many threads doing ranged requests. other libs do this.
> readVectored to call validateNonOverlappingAndReturnSortedRanges before iterating
> this ensures the abfs/s3a requirements are always met, and that because ranges will be read in order, prefetching by other clients will keep their performance good.
> readVectored to add special handling for 0 byte ranges



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org