You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "Tommy Becker (JIRA)" <ji...@apache.org> on 2017/05/15 12:01:04 UTC
[jira] [Commented] (KAFKA-5241) GlobalKTable does not checkpoint
offsets after restoring state
[ https://issues.apache.org/jira/browse/KAFKA-5241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16010389#comment-16010389 ]
Tommy Becker commented on KAFKA-5241:
-------------------------------------
I have a patch for this and will take the issue, if someone can grant me permission to assign to myself.
> GlobalKTable does not checkpoint offsets after restoring state
> --------------------------------------------------------------
>
> Key: KAFKA-5241
> URL: https://issues.apache.org/jira/browse/KAFKA-5241
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 0.10.2.1
> Reporter: Tommy Becker
> Priority: Minor
>
> I'm experimenting with an application that uses a relatively large GlobalKTable, and noticed that streams was not checkpointing its offsets on close(). This is because although {{org.apache.kafka.streams.processor.internals.GlobalStateManagerImpl#restoreState}} updates the checkpoint map, the actual checkpointing itself is guarded by a check that the offsets passed from the {{GloablStateUpdateTask}} are not empty. This is frustrating because if the topic backing the global table is both large (therefore taking a long time to restore) and infrequently written, then streams rebuilds the table from scratch every time the application is started.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)