[jira] [Closed] (FLINK-25458) Support local recovery

["Roman Khachatryan (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLINK-25458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Roman Khachatryan closed FLINK-25458.
-------------------------------------
    Resolution: Fixed

> Support local recovery
> ----------------------
>
>                 Key: FLINK-25458
>                 URL: https://issues.apache.org/jira/browse/FLINK-25458
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Runtime / Checkpointing, Runtime / State Backends
>            Reporter: Yun Tang
>            Assignee: Yanfei Lei
>            Priority: Major
>             Fix For: 1.16.0
>
>
> Currently, changelog state-backend doesn't support local recovery. Thus, recovery times might be sub-optimal.
>  
> Materialized state issues:
> Current periodic materialization would call state backend snapshot method with a materialization id. However, current local state managment would rely on checkpoint id as storing, confirming and discarding. The gap between them would break how local recovery works.
>  
> Non-materialized state issues:
>  * non-materialized state (i.e. changelog) is shared across checkpoints, and therefore needs some tracking (in TM or hard-linking in FS)
>  * the writer does not enforce boundary between checkpoints (when writing to DFS); if local stream simply duplicates DFS stream then it would break on cleanup
>  * files can be shared across tasks, which will also break on cleanup



--
This message was sent by Atlassian Jira
(v8.20.10#820010)