[jira] [Created] (FLINK-35761) Speed up the restore process of unaligned checkpoint

["Rui Fan (Jira)" <ji...@apache.org>]

Rui Fan created FLINK-35761:
-------------------------------

             Summary: Speed up the restore process of unaligned checkpoint
                 Key: FLINK-35761
                 URL: https://issues.apache.org/jira/browse/FLINK-35761
             Project: Flink
          Issue Type: Improvement
          Components: Runtime / Checkpointing
    Affects Versions: 1.19.1, 1.20.0
            Reporter: Rui Fan
            Assignee: Rui Fan


Currently, the task will transition state from ExecutionState.INITIALIZING to ExecutionState.RUNNING after all input buffers are processed.

It will cause the restore time is very long if the performance is not strong and unaligned checkpoint snapshotted too many input buffers. From my experience, the restore time will excess 30 minutes when job with high parallelism.

We hope the job is switched to RUNNING asap. Because the new checkpoint is unable to be triggered during INITIALIZING. If the job is switched to RUNNING, the new unaligned checkpoint can be made.
h2. Brief Solution:
 # The task is switched to RUNNING after all input buffers are added to RecoveredInputChannel.
 ** In general, it's quick unless the network buffer isn't enough.
 # RecoveredInputChannel supports snapshot for network buffers

 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)