You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@flink.apache.org by "Anton Kalashnikov (Jira)" <ji...@apache.org> on 2021/07/21 12:02:00 UTC

[jira] [Created] (FLINK-23452) Measuring subtask throughput

Anton Kalashnikov created FLINK-23452:
-----------------------------------------

             Summary: Measuring subtask throughput
                 Key: FLINK-23452
                 URL: https://issues.apache.org/jira/browse/FLINK-23452
             Project: Flink
          Issue Type: Sub-task
            Reporter: Anton Kalashnikov
            Assignee: Anton Kalashnikov


In the first implementation, throughput could be measured for the whole subtask. The throughput calculation should take into account the numbers of bytes that were handled, the backpressure time and ignore the idle time. The main idea is to keep the balance between idle and backpressure time, so if the backpressure time is high we should decrease the buffer size to provide the configured handling time and vice versa if the subtask is idle time that period should be ignored from calculating the throughput. Otherwise, in the case of network bottleneck, we might have ended up with a small buffer size that’s causing the bottleneck in the first place but we are not able to increase it due to idle time reducing throughput and lowering the buffer size.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)