You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Navneeth Krishnan <re...@gmail.com> on 2019/09/10 06:08:47 UTC

Processor API

Hi All,

I have streaming ETL job which takes data from one kafka topic, enriches
and writes it to another topic. I want to read one more topic which has the
same key and evict the data in state store (added by the processor
function) and send some messages to the same kafka topic. Should I be using
join to achieve this functionality or can I just use the same processor for
two different sources?

Any pointers will be extremely helpful.

Thanks

Re: Processor API

Posted by "Matthias J. Sax" <ma...@confluent.io>.
You can use the same processor. Just define both sources as parents for
the actual processor.

Using the `ProcessorContext` you can distinguish between records from
both topics via `context.topic()` that returns the topic name of the
currently processed record.


-Matthias



On 9/10/19 11:20 PM, Navneeth Krishnan wrote:
> Hi All,
> 
> Any advice on how this could be achieved.
> 
> Thanks
> 
> On Mon, Sep 9, 2019 at 11:08 PM Navneeth Krishnan <re...@gmail.com>
> wrote:
> 
>> Hi All,
>>
>> I have streaming ETL job which takes data from one kafka topic, enriches
>> and writes it to another topic. I want to read one more topic which has the
>> same key and evict the data in state store (added by the processor
>> function) and send some messages to the same kafka topic. Should I be using
>> join to achieve this functionality or can I just use the same processor for
>> two different sources?
>>
>> Any pointers will be extremely helpful.
>>
>> Thanks
>>
>>
>

Re: Processor API

Posted by Navneeth Krishnan <re...@gmail.com>.
Hi All,

Any advice on how this could be achieved.

Thanks

On Mon, Sep 9, 2019 at 11:08 PM Navneeth Krishnan <re...@gmail.com>
wrote:

> Hi All,
>
> I have streaming ETL job which takes data from one kafka topic, enriches
> and writes it to another topic. I want to read one more topic which has the
> same key and evict the data in state store (added by the processor
> function) and send some messages to the same kafka topic. Should I be using
> join to achieve this functionality or can I just use the same processor for
> two different sources?
>
> Any pointers will be extremely helpful.
>
> Thanks
>
>