[jira] [Commented] (PARQUET-2374) Add metrics support for parquet file reader

["Steve Loughran (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/PARQUET-2374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17854370#comment-17854370 ] 

Steve Loughran commented on PARQUET-2374:
-----------------------------------------

I've got reflective access to all the stats stuff in https://github.com/apache/hadoop/pull/6686, but I'm prioritising

# bulk delete in iceberg
# openFile() in parquet

everything is still stabilising there: I can show you the parquet PR if you want, though it's going hand-in-hand with the hadoop one (I've cloned DynMethods there and have the "reference" implementations of reflection based access to the WrappedIO/WrappedStatistics class, for copy and paste into parquet, iceberg &c

> Add metrics support for parquet file reader
> -------------------------------------------
>
>                 Key: PARQUET-2374
>                 URL: https://issues.apache.org/jira/browse/PARQUET-2374
>             Project: Parquet
>          Issue Type: Improvement
>          Components: parquet-mr
>    Affects Versions: 1.13.1
>            Reporter: Parth Chandra
>            Assignee: Parth Chandra
>            Priority: Major
>             Fix For: 1.14.0
>
>
> ParquetFileReader is used by many engines - Hadoop, Spark among them. These engines report various metrics to measure performance in different environments and it is usually useful to be able to get low level metrics out of the file reader and writers.
> It would be very useful to allow a simple interface to report the metrics. Callers can then implement the interface to record the metrics in any subsystem they choose.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org