You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@kafka.apache.org by Javier Holguera <ja...@zopa.com> on 2018/04/25 17:23:03 UTC

KeyValueStore implementation that allows retention policy

Hi,

I have a look around the "state" folder in Kafka Streams and I realised that only WindowStore and SessionStore allows configuring a retention policy.

Looking a bit further, it seems that RocksDbSegmentedBytesStore is the main way to implement a store that can clean itself up based on retention. Internally, this class is pretty much a manager for a collection of Segments. Each Segment is actually a subclass of RocksDbStore.

Would it make sense to create a wrapper that adapts from SegmentedBytesStore to KeyValueStore so we can have an implementation that not only does what RocksDbStore does today, but also allows configuring a retention policy? Is there a reason for this not having been implemented yet?

Thanks.

Re: KeyValueStore implementation that allows retention policy

Posted by Guozhang Wang <wa...@gmail.com>.

Hello Javier,

There are some ticket created for this purpose:

https://issues.apache.org/jira/browse/KAFKA-4212
https://issues.apache.org/jira/browse/KAFKA-4273


Please feel free to share your use cases on the ticket, so that we can
consider prioritizing this feature in Streams.


Guozhang


On Wed, Apr 25, 2018 at 10:23 AM, Javier Holguera <ja...@zopa.com>
wrote:

> Hi,
>
> I have a look around the "state" folder in Kafka Streams and I realised
> that only WindowStore and SessionStore allows configuring a retention
> policy.
>
> Looking a bit further, it seems that RocksDbSegmentedBytesStore is the
> main way to implement a store that can clean itself up based on retention.
> Internally, this class is pretty much a manager for a collection of
> Segments. Each Segment is actually a subclass of RocksDbStore.
>
> Would it make sense to create a wrapper that adapts from
> SegmentedBytesStore to KeyValueStore so we can have an implementation that
> not only does what RocksDbStore does today, but also allows configuring a
> retention policy? Is there a reason for this not having been implemented
> yet?
>
> Thanks.
>



-- 
-- Guozhang