You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@flink.apache.org by "Rui Fan (Jira)" <ji...@apache.org> on 2024/11/21 02:16:00 UTC

[jira] [Created] (FLINK-36753) Adaptive Scheduler actively triggers a Checkpoint

Rui Fan created FLINK-36753:
-------------------------------

             Summary: Adaptive Scheduler actively triggers a Checkpoint
                 Key: FLINK-36753
                 URL: https://issues.apache.org/jira/browse/FLINK-36753
             Project: Flink
          Issue Type: Improvement
          Components: Runtime / Coordination
    Affects Versions: 2.0-preview
            Reporter: Rui Fan


FLIP-461[1] and FLINK-35549[2] support that rescale could be executed after the next completed checkpoint. It greatly reduces the amount of data replay after rescale.

In FLIP-461, Adaptive Scheduler waits for the next periodic checkpoint to be triggered. In most scenarios, a more efficient solution might be Adaptive Scheduler actively triggers a Checkpoint.

The idea comes from an offline discussion between [~mxm]  and me.

[1][https://cwiki.apache.org/confluence/display/FLINK/FLIP-461%3A+Synchronize+rescaling+with+checkpoint+creation+to+minimize+reprocessing+for+the+AdaptiveScheduler]

[2] https://issues.apache.org/jira/browse/FLINK-35549



--
This message was sent by Atlassian Jira
(v8.20.10#820010)