[jira] [Created] (KAFKA-9398) Kafka Streams main thread may not exit even after close timeout has passed

["Bill Bejeck (Jira)" <ji...@apache.org>]

Bill Bejeck created KAFKA-9398:
----------------------------------

             Summary: Kafka Streams main thread may not exit even after close timeout has passed
                 Key: KAFKA-9398
                 URL: https://issues.apache.org/jira/browse/KAFKA-9398
             Project: Kafka
          Issue Type: Improvement
          Components: streams
            Reporter: Bill Bejeck
            Assignee: Bill Bejeck
             Fix For: 2.5.0


Kafka Streams offers the {{KafkaStreams.close()}} method when shutting down a Kafka Streams application.  There are two overloads to this method, one that takes no parameters and another taking a {{Duration}} specifying how long the {{close()}} method should block waiting for streams shutdown operations to complete.  The no-arg version of {{close()}} sets the timeout to {{Long.MAX_VALUE}}.

 

The issue is that if a {{StreamThread}} is some how hung or if one of the {{Consumer}} or {{Producer}} clients are in a hung state, the Kafka Streams application won't exit even after the specified timeout has expired.

For example consider this scenario:
 # A sink topic gets deleted by accident 
 # The {{Producer max.block.ms}} config is set to high value

In this case the {{Producer}} will issue a {{WARN}} logging statement and will continue to make metadata requests looking for the expected topic.  This will continue up until the {{max.block.ms}} expires.  If this value is high enough, calling {{close()}} with a timeout won't fix the issue as when the timeout expires, the Kafka Streams application main thread won't exit.

To prevent this type of issue, we should call {{Thread.interrupt()}} on all {{StreamThread}} instances once the {{close()}} timeout has expired. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)