[jira] [Created] (KAFKA-13350) Handle task corrupted exception on a per state store basis

["Matthias J. Sax (Jira)" <ji...@apache.org>]

Matthias J. Sax created KAFKA-13350:
---------------------------------------

             Summary: Handle task corrupted exception on a per state store basis
                 Key: KAFKA-13350
                 URL: https://issues.apache.org/jira/browse/KAFKA-13350
             Project: Kafka
          Issue Type: Improvement
          Components: streams
            Reporter: Matthias J. Sax


When we hit an `OffsetOutOfRangeException` during restore, we close a tasks as dirty and retry the restore process from scratch. For this case, we wipe out the task's state stores.

If a task has multiple state stores, we also wipe out state that is actually clean and thus need to redo work for no reason. Instead of wiping out all state store, we should only wipe out the single state store that corresponds to the changelog topic partition that hit the `OffsetOutOfRangeException`, but preserve the restore progress for all other state stores.

We need to consider persistent and in-memory stores: for persistent stores, it would be fine to close the not affected stores cleanly and also write the checkpoint file. For in-memory stores however, we should not close the store to avoid dropping the in-memory data.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)