You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Martijn Visser (Jira)" <ji...@apache.org> on 2026/10/23 06:38:01 UTC

[jira] [Resolved] (FLINK-38574) Avoid reusing re-uploaded sst files when checkpoint notification is delayed

     [ https://issues.apache.org/jira/browse/FLINK-38574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Martijn Visser resolved FLINK-38574.
------------------------------------
    Fix Version/s: 1.20.6
       Resolution: Fixed

> Avoid reusing re-uploaded sst files when checkpoint notification is delayed
> ---------------------------------------------------------------------------
>
>                 Key: FLINK-38574
>                 URL: https://issues.apache.org/jira/browse/FLINK-38574
>             Project: Flink
>          Issue Type: Bug
>          Components: Runtime / State Backends
>    Affects Versions: 1.18.1, 1.19.3, 2.0.1, 1.20.3, 2.1.1
>            Reporter: Zakelly Lan
>            Assignee: Zakelly Lan
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 1.19.4, 2.0.2, 2.1.2, 2.2.0, 1.20.6
>
>
> It might be possible thatĀ from the TM's perspective, the checkpoint notification for last checkpoint is executed after the start of the next checkpoint, assuming no concurrent checkpoints. If so, this might indicate a bug. I'm saying that because this may make the `RocksIncrementalSnapshotStrategy` behaves wrong, diverging from the state tracking maintained by the `SharedStateRegistry` on the JM side. For example, considering a buggy event timeline for `RocksIncrementalSnapshotStrategy`:
>  * Checkpoint 1 finished with state handle A for 1.sst.
>  * Checkpoint 2 start, based on no checkpoint, so re-upload the 1.sst with handle B
>  * Received Checkpoint 1's notification of finish.
>  * Checkpoint 2 finished. JM thinks the handle A is subsumed.
>  * Checkpoint 3 start, based on checkpoint 1, for 1.sst we reuse the handle A.
>  * Received Checkpoint 1's subsume, Checkpoint 2's notification of finish.
>  * Checkpoint 3 finished. JM received the handle A's placeholder and tell 'Attempt to reference unknown state', since the handle A is removed when checkpoint 2 finished.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)