Non duplicated WindowStore in Kstream - KStream Join?

[Edmondo Porcu <ed...@gmail.com>]

We need to perform a Kstream - Kstream join with a very large window, where
a tick on the left would trigger a join only with the most recent record on
the right, and viceversa.

This is not how the default window works, since the WindowStoreIterator<V>
returned by window.fetch inside the KStreamKStreamJoinProcessor is an
iterator which can contain multiple records.

In particular, we noticed that the RockDBWindowStore has a retainDuplicates
property set to true, and we would like it to set to false.

How do we customize the store implementation for KStream KStream join?

Re: Non duplicated WindowStore in Kstream - KStream Join?

["Matthias J. Sax" <ma...@confluent.io>]

Question cross-posted at SO:
https://stackoverflow.com/questions/50492491/customize-window-store-implementation-in-kstream-kstream-join

I did put an answer there.

-Matthias

On 5/23/18 8:56 AM, Edmondo Porcu wrote:
> We need to perform a Kstream - Kstream join with a very large window, where
> a tick on the left would trigger a join only with the most recent record on
> the right, and viceversa.
> 
> This is not how the default window works, since the WindowStoreIterator<V>
> returned by window.fetch inside the KStreamKStreamJoinProcessor is an
> iterator which can contain multiple records.
> 
> In particular, we noticed that the RockDBWindowStore has a retainDuplicates
> property set to true, and we would like it to set to false.
> 
> How do we customize the store implementation for KStream KStream join?
>