You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Sachin Mittal <sj...@gmail.com> on 2019/12/06 18:21:28 UTC
Kafka Streams Topology describe format
Hi,
I am just posting a section of my topology to basically understand what
describe method actually displays.
What can we understand just by looking at the topology (like what does -->
and <-- arrows represent).
------------------------------------------------------------------------------------------------------------------------------------------------------------------
Source: KSTREAM-SOURCE-0000000043 (topics:
[price-change-left-repartition])
--> KSTREAM-WINDOWED-0000000047
Source: KSTREAM-SOURCE-0000000046 (topics:
[price-change-right-repartition])
--> KSTREAM-WINDOWED-0000000048
Processor: KSTREAM-WINDOWED-0000000047 (stores:
[KSTREAM-JOINTHIS-0000000049-store])
--> KSTREAM-JOINTHIS-0000000049
<-- KSTREAM-SOURCE-0000000043
Processor: KSTREAM-WINDOWED-0000000048 (stores:
[KSTREAM-OUTEROTHER-0000000050-store])
--> KSTREAM-OUTEROTHER-0000000050
<-- KSTREAM-SOURCE-0000000046
Processor: KSTREAM-JOINTHIS-0000000049 (stores:
[KSTREAM-OUTEROTHER-0000000050-store])
--> KSTREAM-MERGE-0000000051
<-- KSTREAM-WINDOWED-0000000047
Processor: KSTREAM-OUTEROTHER-0000000050 (stores:
[KSTREAM-JOINTHIS-0000000049-store])
--> KSTREAM-MERGE-0000000051
<-- KSTREAM-WINDOWED-0000000048
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------
What I basically understand is that there are 2 sources converted into
windowed source using two of the processors.
There are 2 more subsequent processors which take some input and process
some output.
Ones marked by --> is the input and one marked by <-- is the ouput.
And the string next to Processor: or Source: is the name of the processor
or input of the source.
Is my understand correct or if I am missing something.
Thanks
Sachin
Re: Kafka Streams Topology describe format
Posted by John Roesler <vv...@apache.org>.
Hi again, Sachin,
I highly recommend this tool for helping to understand the topology description: https://zz85.github.io/kafka-streams-viz/
I think your interpretation of the format is pretty much spot-on.
Hope this helps,
-John
On Fri, Dec 6, 2019, at 12:21, Sachin Mittal wrote:
> Hi,
> I am just posting a section of my topology to basically understand what
> describe method actually displays.
>
> What can we understand just by looking at the topology (like what does -->
> and <-- arrows represent).
>
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------
> Source: KSTREAM-SOURCE-0000000043 (topics:
> [price-change-left-repartition])
> --> KSTREAM-WINDOWED-0000000047
> Source: KSTREAM-SOURCE-0000000046 (topics:
> [price-change-right-repartition])
> --> KSTREAM-WINDOWED-0000000048
> Processor: KSTREAM-WINDOWED-0000000047 (stores:
> [KSTREAM-JOINTHIS-0000000049-store])
> --> KSTREAM-JOINTHIS-0000000049
> <-- KSTREAM-SOURCE-0000000043
> Processor: KSTREAM-WINDOWED-0000000048 (stores:
> [KSTREAM-OUTEROTHER-0000000050-store])
> --> KSTREAM-OUTEROTHER-0000000050
> <-- KSTREAM-SOURCE-0000000046
> Processor: KSTREAM-JOINTHIS-0000000049 (stores:
> [KSTREAM-OUTEROTHER-0000000050-store])
> --> KSTREAM-MERGE-0000000051
> <-- KSTREAM-WINDOWED-0000000047
> Processor: KSTREAM-OUTEROTHER-0000000050 (stores:
> [KSTREAM-JOINTHIS-0000000049-store])
> --> KSTREAM-MERGE-0000000051
> <-- KSTREAM-WINDOWED-0000000048
> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> What I basically understand is that there are 2 sources converted into
> windowed source using two of the processors.
> There are 2 more subsequent processors which take some input and process
> some output.
> Ones marked by --> is the input and one marked by <-- is the ouput.
> And the string next to Processor: or Source: is the name of the processor
> or input of the source.
>
> Is my understand correct or if I am missing something.
>
> Thanks
> Sachin
>