[jira] [Assigned] (KAFKA-12634) Should checkpoint after restore finished

["Philip Nee (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-12634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Philip Nee reassigned KAFKA-12634:
----------------------------------

    Assignee: Philip Nee

> Should checkpoint after restore finished
> ----------------------------------------
>
>                 Key: KAFKA-12634
>                 URL: https://issues.apache.org/jira/browse/KAFKA-12634
>             Project: Kafka
>          Issue Type: Improvement
>          Components: streams
>    Affects Versions: 2.5.0
>            Reporter: Matthias J. Sax
>            Assignee: Philip Nee
>            Priority: Critical
>              Labels: new-streams-runtime-should-fix, newbie++
>
> For state stores, Kafka Streams maintains local checkpoint files to track the offsets of the state store changelog topics. The checkpoint is updated on commit or when a task is closed cleanly.
> However, after a successful restore, the checkpoint is not written. Thus, if an instance crashes after restore but before committing, even if the state is on local disk the checkpoint file is missing (indicating that there is no state) and thus state would be restored from scratch.
> While for most cases, the time between restore end and next commit is small, there are cases when this time could be large, for example if there is no new input data to be processed (if there is no input data, the commit would be skipped).
> Thus, we should write the checkpoint file after a successful restore to close this gap (or course, only for at-least-once processing).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)