Kafka Stream custom join using state store

[LuDo <tc...@gmail.com>]

Hi,

I didn't reach any people on StackOverflow, so I try here :
https://stackoverflow.com/questions/67694907/kafka-stream-custom-join-using-state-store

I'm really stuck on that part and I have the feeling only the Processor API
can help me.
But since the stream is really complex, I would prefer a confirmation
before start wasting my time on this.

Can anyone help me about this issue please ?

Thanks

Re: Kafka Stream custom join using state store

[Richard Rossel <he...@gmail.com>]

Hi,
* "... the KTable update won't be trigger on the join on the Transformer."
You are right, KStream-KTable join don't get trigger on KTable update
https://docs.confluent.io/platform/current/streams/developer-guide/dsl-api.html#kstream-ktable-join

Only input records for the left side (stream) trigger the join. Input
records for the right side (table) update only the internal right-side
join state.
From DSL but I think is the same for Processor API

* "is there a way to do a custom join with Kafka Stream DSL ?"
Yes there is, using ValueJoiner
For example, something simple that adds a Object from Ktable to Object on Stream

public class simpleJoiner implements ValueJoiner<classAStream,
classBKtable, classAStream> {
    public classAStream apply(classAStream objFromStream, classBKtable
objFromKtable) {
        objFromStream.setObjB(objFromKtable);
        return objFromStream;
    }
}


On Wed, May 26, 2021 at 3:58 PM LuDo <tc...@gmail.com> wrote:
>
> Hi,
>
> I didn't reach any people on StackOverflow, so I try here :
> https://stackoverflow.com/questions/67694907/kafka-stream-custom-join-using-state-store
>
> I'm really stuck on that part and I have the feeling only the Processor API
> can help me.
> But since the stream is really complex, I would prefer a confirmation
> before start wasting my time on this.
>
> Can anyone help me about this issue please ?
>
> Thanks



-- 
Richard Rossel
Atlanta - GA