You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Dumitru-Nicolae Marasoui <ni...@ovoenergy.com> on 2020/05/11 11:02:16 UTC

compacting to keep oldest values?

Hello kafka community,
As a context, I have to implement a query like
select k, min(d) from topic group by k.

I am wondering if there is a possibility for a compaction to keep the
oldest values, not the newest ones. If this is possible I would create a
new topic with this custom policy to retain oldest values per key and have
a ksql / kstreams pipeline to this new topic.

Alternatively I see a less clear/efficient possibility to create a table
with window aggregation for the oldest values and then at query
time, again, oldest values to be summarized. Here after the first value,
all the subsequent batches on the same keys will add values that are not
useful.

Pls advise,
Thank you
-- 
Thank you,
Nicolae Marasoiu
Scala Engineer
Orion, OVO Group