Clarification regarding multi topics implementation

[Suresh Chidambaram <ch...@gmail.com>]

Hi Team,

Greetings.

I have a use-case wherein I have to consume messages from multiple topics
using Kafka and process it using Kafka Streams,  then publish the message
to multiple target topics.

The example is below.

Source topic A - process A - target topic A
Source topic B - process B - target topic B

Could someone help me achieving this solution?

I have to use Spring Boot with Kafka Streams for this solution.

Thank you.
C Suresh

Re: Clarification regarding multi topics implementation

["Matthias J. Sax" <mj...@apache.org>]

Kafka Streams docs contain a nice tutorial that should help you to get
started: https://kafka.apache.org/25/documentation/streams/tutorial

Not sure if there is anything similar for Spring Boot. I guess your
favorite search engine may help :)


-Matthias

On 4/22/20 9:40 AM, Suresh Chidambaram wrote:
> Hi Team,
> 
> Greetings.
> 
> I have a use-case wherein I have to consume messages from multiple topics
> using Kafka and process it using Kafka Streams,  then publish the message
> to multiple target topics.
> 
> The example is below.
> 
> Source topic A - process A - target topic A
> Source topic B - process B - target topic B
> 
> Could someone help me achieving this solution?
> 
> I have to use Spring Boot with Kafka Streams for this solution.
> 
> Thank you.
> C Suresh
> 

Re: Clarification regarding multi topics implementation

[Suresh Chidambaram <ch...@gmail.com>]

Hi Liam,

Thank you very much for the suggestion.

Thanks
C Suresh

On Thursday, April 23, 2020, Liam Clarke-Hutchinson <
liam.clarke@adscale.co.nz> wrote:

> Hi Suresh,
>
> A Topology can contain several processing workflows. So just create two
> workflows in the topology builder.
>
> StreamsBuilder sb = new StreamsBuilder();
> sb.stream("sourceA").<do A stuff>.to("sinkA");
> sb.stream("sourceB").<do B stuff>.to("sinkB");
> Topology topology = sb.build();
> KafkaStreams streams = KafkaStreams(topology, streamsConfig);
> streams.start();
>
> Hope that helps,
>
> Liam Clarke-Hutchinson
>
> On Thu, 23 Apr. 2020, 4:41 am Suresh Chidambaram, <ch...@gmail.com>
> wrote:
>
> > Hi Team,
> >
> > Greetings.
> >
> > I have a use-case wherein I have to consume messages from multiple topics
> > using Kafka and process it using Kafka Streams,  then publish the message
> > to multiple target topics.
> >
> > The example is below.
> >
> > Source topic A - process A - target topic A
> > Source topic B - process B - target topic B
> >
> > Could someone help me achieving this solution?
> >
> > I have to use Spring Boot with Kafka Streams for this solution.
> >
> > Thank you.
> > C Suresh
> >
>

Re: Clarification regarding multi topics implementation

[Liam Clarke-Hutchinson <li...@adscale.co.nz>]

Hi Suresh,

A Topology can contain several processing workflows. So just create two
workflows in the topology builder.

StreamsBuilder sb = new StreamsBuilder();
sb.stream("sourceA").<do A stuff>.to("sinkA");
sb.stream("sourceB").<do B stuff>.to("sinkB");
Topology topology = sb.build();
KafkaStreams streams = KafkaStreams(topology, streamsConfig);
streams.start();

Hope that helps,

Liam Clarke-Hutchinson

On Thu, 23 Apr. 2020, 4:41 am Suresh Chidambaram, <ch...@gmail.com>
wrote:

> Hi Team,
>
> Greetings.
>
> I have a use-case wherein I have to consume messages from multiple topics
> using Kafka and process it using Kafka Streams,  then publish the message
> to multiple target topics.
>
> The example is below.
>
> Source topic A - process A - target topic A
> Source topic B - process B - target topic B
>
> Could someone help me achieving this solution?
>
> I have to use Spring Boot with Kafka Streams for this solution.
>
> Thank you.
> C Suresh
>