You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Ayoub Omari (Jira)" <ji...@apache.org> on 2024/02/02 13:59:00 UTC

[jira] [Assigned] (KAFKA-15625) Do not flush global state store at each commit

     [ https://issues.apache.org/jira/browse/KAFKA-15625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ayoub Omari reassigned KAFKA-15625:
-----------------------------------

    Assignee: Ayoub Omari

> Do not flush global state store at each commit
> ----------------------------------------------
>
>                 Key: KAFKA-15625
>                 URL: https://issues.apache.org/jira/browse/KAFKA-15625
>             Project: Kafka
>          Issue Type: Improvement
>          Components: streams
>            Reporter: Bruno Cadonna
>            Assignee: Ayoub Omari
>            Priority: Major
>
> Global state stores are flushed at each commit. While that is not a big issue with at-least-once processing mode since the commit interval is by default 30s, it might become an issue with EOS where the commit interval is 200ms by default.
> One option would be to flush and checkpoint global state stores when the delta of the content exceeds a given threshold as we do for other stores. See https://github.com/apache/kafka/blob/a1f3c6d16061566a4f53c72a95e2679b8ee229e0/streams/src/main/java/org/apache/kafka/streams/processor/internals/AbstractTask.java#L97 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)