[jira] [Commented] (KAFKA-5152) Kafka Streams keeps restoring state after shutdown is initiated during startup

["Guozhang Wang (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-5152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16001300#comment-16001300 ] 

Guozhang Wang commented on KAFKA-5152:
--------------------------------------

Thanks [~xvrl] for reporting this issue. I think it is contributed from two issues:

1. When an instance is shutdown, we should be able to interrupt the current on-going restoration process and let it close asap. We can consider adding this support.

2. When the whole app is shutting down, some instances / threads may exit earlier than others. In this case we'd prefer not triggering more rebalances among the rest of the late threads going through the shutdown process. This has been (hopefully) addressed in https://issues.apache.org/jira/browse/KAFKA-4881, in which the exiting consumer will not send the exit group request to coordinator, and hence it will not immediately marked as dead and trigger a rebalance.

> Kafka Streams keeps restoring state after shutdown is initiated during startup
> ------------------------------------------------------------------------------
>
>                 Key: KAFKA-5152
>                 URL: https://issues.apache.org/jira/browse/KAFKA-5152
>             Project: Kafka
>          Issue Type: Bug
>          Components: streams
>    Affects Versions: 0.10.2.1
>            Reporter: Xavier Léauté
>
> If streams shutdown is initiated during state restore (e.g. an uncaught exception is thrown) streams will not shut down until all stores are first finished restoring.
> As restore progresses, stream threads appear to be taken out of service as part of the shutdown sequence, causing rebalancing of tasks. This compounds the problem by slowing down the restore process even further, since the remaining threads now have to also restore the reassigned tasks before they can shut down.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)