[jira] [Updated] (HADOOP-19229) Vector IO on cloud storage: experiment to see what a good minimum seek size should be

["Steve Loughran (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HADOOP-19229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Steve Loughran updated HADOOP-19229:
------------------------------------
    Summary: Vector IO on cloud storage: experiment to see what a good minimum seek size should be  (was: Vector IO: have a max distance between ranges to read)

> Vector IO on cloud storage: experiment to see what a good minimum seek size should be
> -------------------------------------------------------------------------------------
>
>                 Key: HADOOP-19229
>                 URL: https://issues.apache.org/jira/browse/HADOOP-19229
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: 3.4.1
>            Reporter: Steve Loughran
>            Priority: Major
>
> vector iO has a max size to coalesce ranges, but it also needs a maximum gap between ranges to justify the merge. Right now we could have a read where two vectors of size 8 bytes can be merged with a 1 MB gap between them -and that's wasteful. 
> We could also consider an "efficiency" metric which looks at the ratio of bytes-read to bytes-discarded. Not sure what we'd do with it, but we could track it as an IOStat



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org