[jira] [Created] (KAFKA-17120) Race condition in KafkaStreams.close can result in StreamsException "Failed to shut down while in state"

["Andy Coates (Jira)" <ji...@apache.org>]

Andy Coates created KAFKA-17120:
-----------------------------------

             Summary: Race condition in KafkaStreams.close can result in StreamsException "Failed to shut down while in state"
                 Key: KAFKA-17120
                 URL: https://issues.apache.org/jira/browse/KAFKA-17120
             Project: Kafka
          Issue Type: Bug
          Components: streams
    Affects Versions: 3.6.1
            Reporter: Andy Coates


If `KafkaStreams.close` is called while the app is transitioning to an error state, i.e.  `PENDING_ERROR` and `ERROR`, it seems there is a race condition that can result in the `close` method throwing a `StreamsException` with the message "Failed to shut down while in state PENDING_ERROR":

```

[2024-07-02T13:09:05.182Z] 13:09:04.921 [mktx.com.cdc:cdc-processor-b586687d-e969-45eb-949c-09042b0d358b-b76cd6d2-0549-4cf9-b7e5-6f886d408495-StreamThread-1] INFO org.apache.kafka.streams.KafkaStreams - stream-client [mktx.com.cdc:cdc-processor-b586687d-e969-45eb-949c-09042b0d358b-b76cd6d2-0549-4cf9-b7e5-6f886d408495] State transition from RUNNING to PENDING_ERROR

2024-07-02T13:09:05.182Z] 13:09:04.921 [Thread-30] ERROR org.apache.kafka.streams.KafkaStreams - stream-client [cdc:cdc-processor-b586687d-e969-45eb-949c-09042b0d358b-b76cd6d2-0549-4cf9-b7e5-6f886d408495] Failed to transition to PENDING_SHUTDOWN, current state is PENDING_ERROR

[2024-07-02T13:09:05.182Z] 13:09:04.922 [cdc:cdc-processor-b586687d-e969-45eb-949c-09042b0d358b-b76cd6d2-0549-4cf9-b7e5-6f886d408495-StreamThread-1] INFO com.cdc.observability.LoggingApplicationLifecycleListener - \{"state":"PENDING_ERROR","message":"Application-lifecycle","oldState":"RUNNING"}

[2024-07-02T13:09:05.182Z] 13:09:04.922 [Thread-30] ERROR com.cdc.observability.LoggingApplicationLifecycleListener - {"reason":"Failed to stop the Kafka Streams app as streams.close() threw an exception","cause":"org.apache.kafka.streams.errors.StreamsException: Failed to shut down while in state PENDING_ERROR\n\tat org.apache.kafka.streams.KafkaStreams.close(KafkaStreams.java:1447)\n\tat org.apache.kafka.streams.KafkaStreams.close(KafkaStreams.java:1497)\n\tat com.cdc.streams.KafkaStreamsExecutor.shutdownStreams(KafkaStreamsExecutor.java:90)\n\tat com.cdc.streams.KafkaStreamsExecutor.execute(KafkaStreamsExecutor.java:70)\n\tat com.cdc.Main.startKafkaStreams(Main.java:148)\n
```

Could this be caused by lack of synchronisation in the `close` method around the state checks?



--
This message was sent by Atlassian Jira
(v8.20.10#820010)