You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-dev@hadoop.apache.org by "Steve Loughran (Jira)" <ji...@apache.org> on 2023/07/05 13:27:00 UTC

[jira] [Created] (HADOOP-18792) s3a prefetching to use split start/end options to limit prefetch range

Steve Loughran created HADOOP-18792:
---------------------------------------

             Summary: s3a prefetching to use split start/end options to limit prefetch range
                 Key: HADOOP-18792
                 URL: https://issues.apache.org/jira/browse/HADOOP-18792
             Project: Hadoop Common
          Issue Type: Sub-task
            Reporter: Steve Loughran


the bundled hadoop record readers pass the split start/end down to openFile() -these can be used by the prefetching stream as hints as to where to start and stop prefetching.

Hints only as
* records can extend past split end; that last record will still be read when split end < EOF
* for formats with footers which get read, caching that footer is important



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org