You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Jacek Laskowski <ja...@japila.pl> on 2018/09/25 08:56:49 UTC
[STREAMS] Qs on code of State Managers (AbstractStateManager,
ProcessorStateManager and GlobalStateManagerImpl)
Hi Devs,
I'm reviewing state managers and noticed that ProcessorStateManager has
registerGlobalStateStores method that is used exclusively when StreamTask
is created [1] (that seems a bit weird given global state stores
are GlobalStateManagerImpl's actually).
That leads to another question about AbstractStateManager and its two
internal registries stores and globalStores that seem to be managed
by ProcessorStateManager and GlobalStateManagerImpl, respectively (except
the above case when ProcessorStateManager deals with global state store).
Two questions then:
1. Why does ProcessorStateManager do registerGlobalStateStores and register
global state stores (since they are registered and managed elsewhere if
there are global state stores in use)?
2. Why does AbstractStateManager have stores and globalStores since they
should really be in ProcessorStateManager and GlobalStateManagerImpl,
respectively?
There must be something I don't understand yet and hence the questions.
Please help. Thanks.
[1]
https://github.com/apache/kafka/blob/trunk/streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamTask.java#L240
Pozdrawiam,
Jacek Laskowski
----
https://about.me/JacekLaskowski
Mastering Spark SQL https://bit.ly/mastering-spark-sql
Spark Structured Streaming https://bit.ly/spark-structured-streaming
Mastering Kafka Streams https://bit.ly/mastering-kafka-streams
Follow me at https://twitter.com/jaceklaskowski
Re: [STREAMS] Qs on code of State Managers (AbstractStateManager,
ProcessorStateManager and GlobalStateManagerImpl)
Posted by Guozhang Wang <wa...@gmail.com>.
Hello Jacek,
Your observation is valid, currently the code hierarchy of
AbstractStateManager, ProcessorStateManager and GlobalStateManagerImpl are
a bit intrusive to each other and would better be cleaned a bit.
Feel free to create a JIRA with your observed placed to clean up and submit
a PR.
Guozhang
On Tue, Sep 25, 2018 at 1:56 AM, Jacek Laskowski <ja...@japila.pl> wrote:
> Hi Devs,
>
> I'm reviewing state managers and noticed that ProcessorStateManager has
> registerGlobalStateStores method that is used exclusively when StreamTask
> is created [1] (that seems a bit weird given global state stores
> are GlobalStateManagerImpl's actually).
>
> That leads to another question about AbstractStateManager and its two
> internal registries stores and globalStores that seem to be managed
> by ProcessorStateManager and GlobalStateManagerImpl, respectively (except
> the above case when ProcessorStateManager deals with global state store).
>
> Two questions then:
>
> 1. Why does ProcessorStateManager do registerGlobalStateStores and register
> global state stores (since they are registered and managed elsewhere if
> there are global state stores in use)?
>
> 2. Why does AbstractStateManager have stores and globalStores since they
> should really be in ProcessorStateManager and GlobalStateManagerImpl,
> respectively?
>
> There must be something I don't understand yet and hence the questions.
> Please help. Thanks.
>
> [1]
> https://github.com/apache/kafka/blob/trunk/streams/src/
> main/java/org/apache/kafka/streams/processor/internals/
> StreamTask.java#L240
>
> Pozdrawiam,
> Jacek Laskowski
> ----
> https://about.me/JacekLaskowski
> Mastering Spark SQL https://bit.ly/mastering-spark-sql
> Spark Structured Streaming https://bit.ly/spark-structured-streaming
> Mastering Kafka Streams https://bit.ly/mastering-kafka-streams
> Follow me at https://twitter.com/jaceklaskowski
>
--
-- Guozhang