[jira] [Assigned] (FLINK-39835) AsyncWaitOperator supports soft backpressure to avoid blocking unaligned checkpoint

["Rui Fan (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLINK-39835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Rui Fan reassigned FLINK-39835:
-------------------------------

    Assignee: Gen Luo

> AsyncWaitOperator supports soft backpressure to avoid blocking unaligned checkpoint
> -----------------------------------------------------------------------------------
>
>                 Key: FLINK-39835
>                 URL: https://issues.apache.org/jira/browse/FLINK-39835
>             Project: Flink
>          Issue Type: Improvement
>          Components: Runtime / Task
>            Reporter: Gen Luo
>            Assignee: Gen Luo
>            Priority: Major
>              Labels: pull-request-available
>
> When AsyncWaitOperator processes input, if the async queue is full, it yields and waits for a callback to complete to free up a slot. During this time, unaligned checkpoints cannot be processed.
> Soft backpressure is used to ensure UC can be processed in time. In the non-async path, processInput produces data written to the ResultPartition. If the RP is full, processInput can be blocked by soft backpressure so that mails can be executed.
> The same idea can apply to AsyncWaitOperator: on the async path, processInput produces requests written to the AsyncWaitOperator's queue — when the queue is full, processInput should be blocked. Similarly, results from AsyncWaitOperator produce data written to downstream (e.g., RP) — if downstream is full, AsyncWaitOperator's emitResult should also be blocked.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)