[jira] [Created] (FLINK-34974) FLIP-424: Asynchronous State APIs

["Zakelly Lan (Jira)" <ji...@apache.org>]

Zakelly Lan created FLINK-34974:
-----------------------------------

             Summary: FLIP-424: Asynchronous State APIs
                 Key: FLINK-34974
                 URL: https://issues.apache.org/jira/browse/FLINK-34974
             Project: Flink
          Issue Type: New Feature
          Components: Runtime / State Backends
            Reporter: Zakelly Lan
            Assignee: Zakelly Lan
             Fix For: 2.0.0


This is a sub-FLIP for the disaggregated state management and its related work, please read the [FLIP-423|https://cwiki.apache.org/confluence/x/R4p3EQ] first to know the whole story.

To maximize I/O capacity utilization and enhance the use of pre-allocated computational resources, this FLIP proposes the introduction of asynchronous state APIs. These APIs permit state access to be executed in threads separate from the task thread, returning the result when available. Consequently, the task thread can process another element while awaiting multiple pending state results. This enables concurrent processing of multiple records, ensuring that the latency of individual I/O operations no longer has a direct impact on job performance. This approach is particularly advantageous in scenarios where I/O bandwidth is underutilized and I/O latency is the limiting factor. The Disaggregated Storage Architecture, as discussed in FLIP-423, is a prime example of a scenario characterized by abundant and easily scalable I/O bandwidth coupled with higher I/O latency. The asynchronous state APIs hold great promise for significantly enhancing Flink's performance when dealing with disaggregated state.




--
This message was sent by Atlassian Jira
(v8.20.10#820010)