You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@flink.apache.org by "Feifan Wang (Jira)" <ji...@apache.org> on 2023/01/26 02:08:00 UTC

[jira] [Created] (FLINK-30792) clean up not uploaded state changes after materialization complete

Feifan Wang created FLINK-30792:
-----------------------------------

             Summary: clean up not uploaded state changes after materialization complete
                 Key: FLINK-30792
                 URL: https://issues.apache.org/jira/browse/FLINK-30792
             Project: Flink
          Issue Type: Bug
          Components: Runtime / State Backends
    Affects Versions: 1.16.0
            Reporter: Feifan Wang


We should clean up not uploaded state changes after materialization completed, otherwise it will cause (status quo) : 
 # subsequent checkpoints contain wrong state changes which before completed materialization
 # FileNotFound exception may occur when recovering from the above problematic checkpoint, because the state change files before completed materialization may have been deleted with the checkpoint subsuming.

Since state changes before completed materialization in FsStateChangelogWriter#notUploaded will not be used in any subsequent checkpoint, I suggest clean up it while handle materialization result. 

How do you think about this ? [~ym] , [~roman] 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)