You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Roman Khachatryan (Jira)" <ji...@apache.org> on 2022/01/05 14:16:00 UTC

[jira] [Updated] (FLINK-23252) Support recovery (from checkpoints) after disabling changelog backend

     [ https://issues.apache.org/jira/browse/FLINK-23252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Roman Khachatryan updated FLINK-23252:
--------------------------------------
    Description: 
Recovery from savepoint already works:
 * changelog off -> on: FLINK-23278
 * changelog on -> off: directly calling {{{}underlyingBackend.{}}}{{{}savepoint{}}}{{{}(){}}} (instead of {{{}writer.persist(){}}}) 

 

For checkpoints, changelog on -> off doesn't work. Regular state backend will receive materialized and non-aterialized state on recovery. To support it, state changes have to be applied on recovery even if changelog backend is disabled.

Another obstacle could be that wrapped backend state handles created during materialization can refer to a fake checkpoint ID. This fake checkpoint ID would then be passed to e.g. RocksIncrementalSnapshotStrategy on restore.

  was:
Recovery from savepoint already works:
 * changelog off -> on: FLINK-23278
 * changelog on -> off: directly calling {{underlyingBackend.}}{{savepoint}}{{()}} (instead of {{writer.persist()}}) 

For checkpoints, changelog on -> off doesn't work. Regular state backend will receive materialized and non-aterialized state on recovery. To support it, state changes have to be applied on recovery even if changelog backend is disabled.


> Support recovery (from checkpoints) after disabling changelog backend
> ---------------------------------------------------------------------
>
>                 Key: FLINK-23252
>                 URL: https://issues.apache.org/jira/browse/FLINK-23252
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Runtime / State Backends
>            Reporter: Roman Khachatryan
>            Priority: Major
>             Fix For: 1.15.0
>
>
> Recovery from savepoint already works:
>  * changelog off -> on: FLINK-23278
>  * changelog on -> off: directly calling {{{}underlyingBackend.{}}}{{{}savepoint{}}}{{{}(){}}} (instead of {{{}writer.persist(){}}}) 
>  
> For checkpoints, changelog on -> off doesn't work. Regular state backend will receive materialized and non-aterialized state on recovery. To support it, state changes have to be applied on recovery even if changelog backend is disabled.
> Another obstacle could be that wrapped backend state handles created during materialization can refer to a fake checkpoint ID. This fake checkpoint ID would then be passed to e.g. RocksIncrementalSnapshotStrategy on restore.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)