[jira] [Commented] (KAFKA-13680) Kafka Streams application remains in RUNNING state although all stream threads shut down

["Bruno Cadonna (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17496051#comment-17496051 ] 

Bruno Cadonna commented on KAFKA-13680:
---------------------------------------

Hi [~denisw], Thank you for the report!

Did you set an exception handler with the following deprecated method?

{code:java}
public void setUncaughtExceptionHandler(final Thread.UncaughtExceptionHandler uncaughtExceptionHandler)
{code} 

Since 2.8, a thread death should lead to the shutdown of the client by default. That is, if no custom exception handler is set with 

{code:java}
public void setUncaughtExceptionHandler(final StreamsUncaughtExceptionHandler userStreamsUncaughtExceptionHandler)
{code}

If a custom handler is set then the dying thread is either replaced (i.e., processing continues), the client is shutdown, or the whole Streams application is shutdown. The situation where single stream threads die and processing continues with less stream threads should never happen -- except, you use the deprecated exception handler.

The Kafka Streams instance stays in RUNNING, because with 2.8 we also introduced APIs to add and remove stream threads during execution and now the case that a Kafka Streams instance has zero stream threads running is a non-error state.

So if you have the old deprecated exception handler set, I would recommend to use the new one. 

> Kafka Streams application remains in RUNNING state although all stream threads shut down
> ----------------------------------------------------------------------------------------
>
>                 Key: KAFKA-13680
>                 URL: https://issues.apache.org/jira/browse/KAFKA-13680
>             Project: Kafka
>          Issue Type: Bug
>          Components: streams
>    Affects Versions: 3.0.0
>            Reporter: Denis Washington
>            Priority: Major
>
> We have a Kafka Streams application that is configured with {{LogAndFailExceptionHandler}} as deserialization error handler. In the Kafka Streams version we used previously (2.7), a deserialization error that causes all stream threads to shut down would ultimately move the application to the ERROR state.
> However, after updating to Kafka Streams 3.0.0, we see a different behavior: the stream threads still shut down, but the Kafka Streams application stays in the RUNNING state. It  thus gets into a "zombie" state not detected by our monitoring.
> It may be worth noting that this application has global state stores, and that the global stream thread was not affected by the deserialization error where we noticed the problem.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)