You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@flink.apache.org by "Xintong Song (Jira)" <ji...@apache.org> on 2021/01/15 00:58:02 UTC

[jira] [Updated] (FLINK-20465) Fail globally when not resuming from the latest checkpoint in regional failover

     [ https://issues.apache.org/jira/browse/FLINK-20465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Xintong Song updated FLINK-20465:
---------------------------------
    Fix Version/s:     (was: 1.12.1)
                   1.12.2

> Fail globally when not resuming from the latest checkpoint in regional failover
> -------------------------------------------------------------------------------
>
>                 Key: FLINK-20465
>                 URL: https://issues.apache.org/jira/browse/FLINK-20465
>             Project: Flink
>          Issue Type: Improvement
>          Components: Runtime / Coordination
>    Affects Versions: 1.12.0
>            Reporter: Till Rohrmann
>            Priority: Critical
>             Fix For: 1.13.0, 1.12.2
>
>
> As a follow up for FLINK-20290 we should assert that we resume from the latest checkpoint when doing a regional failover in the {{SourceCoordinators}} in order to avoid losing input splits (see FLINK-20427). If the assumption does not hold, then we should fail the job globally so that we reset the master state to a consistent view of the state. Such a behaviour can act as a safety net in case that Flink ever tries to recover from not the latest available checkpoint.
> One idea how to solve it is to remember the latest completed checkpoint id somewhere along the way to the {{SplitAssignmentTracker.getAndRemoveUncheckpointedAssignment}} and failing when the restored checkpoint id is smaller.
> cc [~sewen], [~jqin]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)