[jira] [Updated] (HADOOP-17273) FSDataInput/Output Streams to automatically collect IOStatistics

["Steve Loughran (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HADOOP-17273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Steve Loughran updated HADOOP-17273:
------------------------------------
    Parent: HADOOP-17469  (was: HADOOP-16830)

> FSDataInput/Output Streams to automatically collect IOStatistics
> ----------------------------------------------------------------
>
>                 Key: HADOOP-17273
>                 URL: https://issues.apache.org/jira/browse/HADOOP-17273
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs
>    Affects Versions: 3.3.1
>            Reporter: Steve Loughran
>            Priority: Major
>
> the fs input/output streams to automatically collect stream IO statistics even if the inner classes don't; count invocations and durations
> An issue here becomes "how would you aggregate with the inner stream, *efficiently*. Not sure there, except maybe
> * have some interface to access an updatable IOStatisticsStore which, if the stream implements, says "here is something you can update"
> * wrapper class updates that with counts, durations
> * we extend DynamicIOStatistics for each counter/gauge etc to have a factory which can add new entries on demand (new AtomicLong, etc)
> * so wrapper classes just update their stats, which  updates existing stats or triggers off an on-demand creation of a new entry
> Not so informative in terms of low level details (HTTP/IPC requests and latency, errors, bytes discarded) but would give callers benefits of using the API for HDFS, Ozone, GCS



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org