[jira] [Updated] (FLINK-19537) Processed in-flight bytes metric is not accurate

["Arvid Heise (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLINK-19537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Arvid Heise updated FLINK-19537:
--------------------------------
    Fix Version/s: 1.13.0

> Processed in-flight bytes metric is not accurate
> ------------------------------------------------
>
>                 Key: FLINK-19537
>                 URL: https://issues.apache.org/jira/browse/FLINK-19537
>             Project: Flink
>          Issue Type: Improvement
>          Components: Runtime / Metrics, Runtime / Network
>    Affects Versions: 1.12.0
>            Reporter: Piotr Nowojski
>            Priority: Minor
>             Fix For: 1.13.0
>
>
> Processed in-flight bytes as introduced in FLINK-18662 is not entirely accurate, as it's ignoring the buffer/bytes accumulated in the record deserializers. If buffer is processed here, it doesn't mean it was fully processed (so we can over estimate the amount of processed bytes). On the other hand some records/bytes might be processed without polling anything from this {{CheckpointedInputGate}} (underestimating the amount of processed bytes). All in all this should have been calculated on the {{StreamTaskNetworkInput}} level, where we have an access to the records deserializers. However the current is on average accurate and it might be just good enough (at least for the time being).
> Also this metric is currently ignoring chained source inputs to the multiple input stream task.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)