You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@flink.apache.org by "zhijiang (Jira)" <ji...@apache.org> on 2019/10/21 07:28:00 UTC

[jira] [Created] (FLINK-14472) Implement back-pressure monitor with non-blocking outputs

zhijiang created FLINK-14472:
--------------------------------

             Summary: Implement back-pressure monitor with non-blocking outputs
                 Key: FLINK-14472
                 URL: https://issues.apache.org/jira/browse/FLINK-14472
             Project: Flink
          Issue Type: Task
          Components: Runtime / Network
            Reporter: zhijiang
             Fix For: 1.10.0


Currently back-pressure monitor relies on detecting task threads that are stuck in `requestBufferBuilderBlocking`. There are actually two cases to cause back-pressure ATM:
 * There are no available buffers in `LocalBufferPool` and all the given quotas from global pool are also exhausted. Then we need to wait for buffer recycling to `LocalBufferPool`.
 * No available buffers in `LocalBufferPool`, but the quota has not been used up. While requesting buffer from global pool, it is blocked because of no available buffers in global pool. Then we need to wait for buffer recycling to global pool.

We already implemented the non-blocking output for the first case in [FLINK-14396|https://issues.apache.org/jira/browse/FLINK-14396], and we expect the second case done together with adjusting the back-pressure monitor which could check for `RecordWriter#isAvailable` instead.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)