You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "John Roesler (Jira)" <ji...@apache.org> on 2022/04/07 03:30:00 UTC
[jira] [Updated] (KAFKA-13622) Revisit the complexity of position tracking in state stores
[ https://issues.apache.org/jira/browse/KAFKA-13622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Roesler updated KAFKA-13622:
---------------------------------
Labels: IQv2 (was: )
> Revisit the complexity of position tracking in state stores
> -----------------------------------------------------------
>
> Key: KAFKA-13622
> URL: https://issues.apache.org/jira/browse/KAFKA-13622
> Project: Kafka
> Issue Type: Sub-task
> Reporter: John Roesler
> Priority: Minor
> Labels: IQv2
>
> Currently, state store implementers have a significant burden to track position correctly. They have to:
> * update the position during all puts
> * implement the RecordBatchingStateRestoreCallback and use the {color:#000000}ChangelogRecordDeserializationHelper to update the position based on record headers{color}
> * {color:#000000}implement some mechanism to restore the position after a restart if the store is persistent (such as supply a CommitCallback to write the position to a local file and then read the file during init){color}
> {color:#000000}[~guozhang] pointed out during review that this is probably too much responsibility (and certainly too much opportunity for error). We should see what we can do to simplify these responsibilities, if not eliminate them entirely from the store implementer's scope of concern.
> {color}
>
> {color:#000000}See https://github.com/apache/kafka/pull/11676#discussion_r790358058{color}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)