You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Rainer Guessner <ra...@gmx.com> on 2017/11/09 17:38:25 UTC

Kafka Streams - Custom processor "init" method called before state store has data restored into it

I have a custom processor that implements AbstractProcessor and a custom store that implements StateStore.
 
Before Kafka 1.0.0 the processors "init" method gets called after the state store is restored from changelog and that is good.
With Kafka 1.0.0 the processors "init" method is called BEFORE the state store is restored from changelog and that is bad.
 
My processor can only initialize when it has access to the state. However at the time KStreams calls "init" on the processor the state store may not have any data. It is not an option for me to initialize the processor lazily when a record arrives, or to re-initialize it when "onRestoreEnd" is called (its only called on restore; The state store "init" gets called before processor "init" regardless of restore or not.)
 
I think I need to have either of these:
a) know whether or not a state restore will take place and when not
b) or get a call to the state store regardless of whether state restore took place or not
c) or I need a "ready" method on the processor that gets called when the state store has completed restoring and is actually usable
 
Please help, thank you in advance.
Rainer