You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@flink.apache.org by "Anton Kalashnikov (Jira)" <ji...@apache.org> on 2021/09/07 09:40:00 UTC

[jira] [Created] (FLINK-24189) Debloating buffer for multiply gates

Anton Kalashnikov created FLINK-24189:
-----------------------------------------

             Summary: Debloating buffer for multiply gates
                 Key: FLINK-24189
                 URL: https://issues.apache.org/jira/browse/FLINK-24189
             Project: Flink
          Issue Type: Sub-task
          Components: Runtime / Network
    Affects Versions: 1.14.0
            Reporter: Anton Kalashnikov
             Fix For: 1.15.0


Right now, the buffer debloat suppose that works with SingleInputGates which have a similar load. It needs to support UnionInputGate in case of data skew. 
The possible implementation can be the calculation of the throughput separately for each gate or calculation of the total throughput but choosing the buffer size independently for each gate based on their buffers in use.

It is highly important to fairly share the throughput among all gates. In other words, avoid the situation:
* gate1 has a low load while gate2 has a high load
* the small buffer size was set for gate1 and the big buffer size for gate2
* the load for gate1 increased up to the load of gate2
* it is impossible to increase the buffer size for gate1 because it is no reason to decrease the buffer size for gate2 since the load for it doesn't change



--
This message was sent by Atlassian Jira
(v8.3.4#803005)