You are viewing a plain text version of this content. The canonical link for it is here.

Posted to jira@arrow.apache.org by "Neville Dipale (Jira)" <ji...@apache.org> on 2021/03/30 16:35:00 UTC

[jira] [Created] (ARROW-12153) [Rust] [Parquet] Return file metadata after writing Parquet file

Neville Dipale created ARROW-12153:
--------------------------------------

             Summary: [Rust] [Parquet] Return file metadata after writing Parquet file
                 Key: ARROW-12153
                 URL: https://issues.apache.org/jira/browse/ARROW-12153
             Project: Apache Arrow
          Issue Type: New Feature
            Reporter: Neville Dipale
            Assignee: Neville Dipale


Parquet writers like delta-rs rely on the Parquet metadata to write file-level statistics for file pruning purposes.

We currently do not expose these stats, requiring the writer to read the file that has just been written, to get the stats. This is more problematic for in-memory sinks, as there is currently no way of getting the metadata from the sink before it's persisted.

Explore if we can expose these stats to the writer, to make the above easier.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)