Kafka Streams - StateRestoreListener called when new partitions assigned

[Javier Holguera <ja...@gmail.com>]

Hi,

I understand that the state store listener that can be set using
KafkaStreams.setGlobalStateRestoreListener will be invoked when the streams
app starts if it doesn't find the state locally (e.g., running on a
ephemeral docker container).

However, I wonder if the process happens as well if there is a rebalance
and the streams app gets assigned new partitions for which it doesn't have
state yet. Does the app go into the rebalancing state and stays there until
the state for those new partitions have been restored? Or does it go back
to running even if it hasn't happened yet?

Thanks.

Regards,
Javier.

Re: Kafka Streams - StateRestoreListener called when new partitions assigned

[Guozhang Wang <wa...@gmail.com>]

Hello Javier,

When a rebalance happened and the new tasks (hence input partitions) are
assigned that need to be restored, the state of the instance would also
transit to REBALANCING, and would only be transit back to RUNNING after all
tasks have been completed restoring and all are being processed normally.

Looking at the docs, I think we can add a specific paragraph on web docs
"developer guide" on exactly when the state transition would happen to
clarify those things, could you create a JIRA ticket for improving on our
docs so that we would not drop this on the floor?

Guozhang

On Tue, Nov 12, 2019 at 4:16 AM Javier Holguera <ja...@gmail.com>
wrote:

> Hi,
>
> I understand that the state store listener that can be set using
> KafkaStreams.setGlobalStateRestoreListener will be invoked when the streams
> app starts if it doesn't find the state locally (e.g., running on a
> ephemeral docker container).
>
> However, I wonder if the process happens as well if there is a rebalance
> and the streams app gets assigned new partitions for which it doesn't have
> state yet. Does the app go into the rebalancing state and stays there until
> the state for those new partitions have been restored? Or does it go back
> to running even if it hasn't happened yet?
>
> Thanks.
>
> Regards,
> Javier.
>


-- 
-- Guozhang