[jira] [Reopened] (KAFKA-18067) Kafka Streams can leak Producer client under EOS

["Bruno Cadonna (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-18067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Bruno Cadonna reopened KAFKA-18067:
-----------------------------------

We had to revert the fix for this bug (https://github.com/apache/kafka/pull/19078) because it introduces a blocking bug for AK 4.0. 
The issue is that the fix prevented Kafka Streams from re-initializing its transactional producer under exactly-once semantics. That led to an infinite loop of {{ProducerFencedException}}s with corresponding rebalances.

For example:
# 1 A network partitions happens that causes the timeout of a transaction.
# 2 The transactional producer is fenced due to invalid producer epoch.
# 3 Kafka Streams closes the tasks dirty and re-joins the group, i.e., a rebalance is triggered. 
# 4 The transactional producer is NOT re-initialized and does NOT get a new producer epoch. 
# 5 Processing starts but the transactional producer is immediately fenced during the attempt to start a new transaction because of the invalid producer epoch.
# step 3 is repeated

> Kafka Streams can leak Producer client under EOS
> ------------------------------------------------
>
>                 Key: KAFKA-18067
>                 URL: https://issues.apache.org/jira/browse/KAFKA-18067
>             Project: Kafka
>          Issue Type: Bug
>          Components: streams
>            Reporter: A. Sophie Blee-Goldman
>            Assignee: TengYao Chi
>            Priority: Major
>              Labels: newbie, newbie++
>             Fix For: 4.0.0
>
>
> Under certain conditions Kafka Streams can end up closing a producer client twice and creating a new one that then is never closed.
> During a StreamThread's shutdown, the TaskManager is closed first, through which the thread's producer client is also closed. Later on we call #unsubscribe on the main consumer, which can result in the #onPartitionsLost callback being invoked and ultimately trying to reset/reinitialize the StreamsProducer if EOS is enabled. This in turn includes closing the current producer and creating a new one. And since the current producer was already closed, we end up closing that client twice and never closing the newly created producer.
> Ideally we would just skip the reset/reinitialize process entirely when invoked during shutdown. This solves the two problems here (leaked client and double close), while also removing the unnecessary overhead of creating an entirely new client just to throw it away



--
This message was sent by Atlassian Jira
(v8.20.10#820010)