[jira] [Resolved] (KAFKA-8204) Streams may flush state stores in the incorrect order

["Guozhang Wang (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-8204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Guozhang Wang resolved KAFKA-8204.
----------------------------------
       Resolution: Fixed
    Fix Version/s:     (was: 2.1.2)
                       (was: 2.0.2)
                       (was: 1.1.2)
                   2.3.0

The latest PR has been merged to 2.2 / trunk, we will add more to fixed versions as we cherry-pick it to older branches via different PRs.

> Streams may flush state stores in the incorrect order
> -----------------------------------------------------
>
>                 Key: KAFKA-8204
>                 URL: https://issues.apache.org/jira/browse/KAFKA-8204
>             Project: Kafka
>          Issue Type: Improvement
>          Components: streams
>    Affects Versions: 1.1.0, 1.1.1, 2.0.0, 2.0.1, 2.1.0, 2.2.0, 2.1.1
>            Reporter: John Roesler
>            Assignee: John Roesler
>            Priority: Blocker
>             Fix For: 2.3.0, 2.2.1
>
>
> Cached state stores may forward records during a flush call, so Streams should flush the stores in topological order. Otherwise, Streams may flush a downstream store before an upstream one, resulting in sink results being committed without the corresponding state changelog updates being committed.
> This behavior is partly responsible for the bug reported in KAFKA-7895 .
> The fix is simply to flush the stores in topological order, then when the upstream store forwards records to a downstream stateful processor, the corresponding state changes will be correctly flushed as well.
> An alternative would be to repeatedly call flush on all state stores until they report there is nothing left to flush, but this requires a public API change to enable state stores to report whether they need a flush or not.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)