[jira] [Resolved] (KAFKA-15116) Kafka Streams processing blocked during rebalance

["A. Sophie Blee-Goldman (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-15116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

A. Sophie Blee-Goldman resolved KAFKA-15116.
--------------------------------------------
    Resolution: Not A Problem

> Kafka Streams processing blocked during rebalance
> -------------------------------------------------
>
>                 Key: KAFKA-15116
>                 URL: https://issues.apache.org/jira/browse/KAFKA-15116
>             Project: Kafka
>          Issue Type: Bug
>          Components: streams
>    Affects Versions: 3.5.0
>            Reporter: David Gammon
>            Priority: Major
>
> We have a Kafka Streams application that simply takes a messages, processes it and then produces an event out the other side. The complexity is that there is a requirement that all events with the same partition key must be committed before the next message  is processed.
> This works most of the time flawlessly but we have started to see problems during deployments where the first message blocks the second message during a rebalance because the first message isn’t committed before the second message is processed. This ultimately results in transactions timing out and more rebalancing.
> We’ve tried lots of configuration to get the behaviour we require with no luck. We’ve now put in a temporary fix so that Kafka Streams works with our framework but it feels like this might be a missing feature or potentially a bug.
> +Example+
> Given:
>  * We have two messages (InA and InB).
>  * Both messages have the same partition key.
>  * A rebalance is in progress so streams is no longer able to commit.
> When:
>  # Message InA -> processor -> OutA (not committed)
>  # Message InB -> processor -> blocked because #1 has not been committed



--
This message was sent by Atlassian Jira
(v8.20.10#820010)