[GitHub] [arrow-rs] wjones127 opened a new issue, #4315: Consider using read_at for local read_range

["wjones127 (via GitHub)" <gi...@apache.org>]

wjones127 opened a new issue, #4315:
URL: https://github.com/apache/arrow-rs/issues/4315

   **Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
   
   Right now in `read_range`, we open the file, seek to the start of the range, and then read. However depending on the access pattern, it might make more sense to use [std::os::unix::fs::FileExt::read_at](https://doc.rust-lang.org/std/os/unix/fs/trait.FileExt.html) (there is a Windows equivalent as well). This uses the [pread64]() syscall under the hood, which reads at an offset without moving the file cursor.
   
   **Describe the solution you'd like**
   
   We should benchmark this against the current access pattern. In our recent tests we find it's 2 - 3 times faster for random reads of 4k bytes.
   
   **Describe alternatives you've considered**
   
   There may be other ideas, or we may find in benchmarks this isn't ideal for many other common read patterns (sequential reads).
   
   **Additional context**
   <!--
   Add any other context or screenshots about the feature request here.
   -->
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow-rs] wjones127 closed issue #4315: Consider using read_at for local read_range

["wjones127 (via GitHub)" <gi...@apache.org>]

wjones127 closed issue #4315: Consider using read_at for local read_range
URL: https://github.com/apache/arrow-rs/issues/4315


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org