You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@arrow.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2020/10/26 08:11:00 UTC

[jira] [Updated] (ARROW-10387) [Rust] Avoid call for file size metadata when reading parquet footer

     [ https://issues.apache.org/jira/browse/ARROW-10387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

ASF GitHub Bot updated ARROW-10387:
-----------------------------------
    Labels: pull-request-available  (was: )

> [Rust] Avoid call for file size metadata when reading parquet footer
> --------------------------------------------------------------------
>
>                 Key: ARROW-10387
>                 URL: https://issues.apache.org/jira/browse/ARROW-10387
>             Project: Apache Arrow
>          Issue Type: Improvement
>            Reporter: Remi Dettai
>            Priority: Major
>              Labels: pull-request-available
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> This is an extension to [ARROW-10368|https://issues.apache.org/jira/browse/ARROW-10368].
> When reading a Parquet file, you first need to read its footer and metadata. If you only have "read from start" capability, this means you need the size of the file to read relatively to end. On some storages, getting the size metadata can be expensive (e.g extra http call for blob storage).
> The proposition is to add the capability to "read from end" to the ChunkReader trait as most file storages will have this feature (file storage as well as blob storages).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)