You are viewing a plain text version of this content. The canonical link for it is here.

Posted to jira@kafka.apache.org by "Richard Yu (JIRA)" <ji...@apache.org> on 2018/12/16 04:00:00 UTC

[jira] [Commented] (KAFKA-3184) Add Checkpoint for In-memory State Store

    [ https://issues.apache.org/jira/browse/KAFKA-3184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16722380#comment-16722380 ] 

Richard Yu commented on KAFKA-3184:
-----------------------------------

As Ted pointed out, persistent() in {{InMemoryKeyValueStore}} would have to return true with this patch. So then would other stores which formerly returned false for persistent() also be committed? 

> Add Checkpoint for In-memory State Store
> ----------------------------------------
>
>                 Key: KAFKA-3184
>                 URL: https://issues.apache.org/jira/browse/KAFKA-3184
>             Project: Kafka
>          Issue Type: Improvement
>          Components: streams
>            Reporter: Guozhang Wang
>            Priority: Major
>              Labels: user-experience
>
> Currently Kafka Streams does not make a checkpoint of the persistent state store upon committing, which would be expensive since it is "stopping the world" and write on disks: for example, RocksDB would require you to copy the file directory to make a copy naively. 
> However, for in-memory stores checkpointing maybe doable in an asynchronous manner hence it can be done quickly. And the benefit of having intermediate checkpoint is to avoid restoring from scratch if standby tasks are not present.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)