You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Miguel González <mi...@klar.mx> on 2021/12/24 19:08:16 UTC
How to properly use a clean a TimestampedKeyValueStore
Hello
I'm using Kafka Streams and I have a transformer that uses
a TimestampedKeyValueStore, I have a punctuator that is in charge of
cleaning the store,
Basically I'm iterating the store using kvStore.all() and deleting the keys
based on some logic with kvStore.delete(key);
I'm seeing the changelog topic for the store grow unbounded, I'm seeing
many values with null for the same keys... I think those are called
tombstones right? but the punctuator is constantly doing the same thing
trying to delete the same keys.. I see more tombstones being inserted.
Is this the expected behavior? If so, how can I correctly clean that store?
thanks
- Miguel
Re: How to properly use a clean a TimestampedKeyValueStore
Posted by Guozhang Wang <wa...@gmail.com>.
Hi Miguel,
I checked your code and it seems fine to me, so I would not suspect
anything with your logic. The next thing I'd suggest to check if you have
many cases where the same key gets deleted and then re-inserted (you can
add some logging at the `put` and `delete` calls`).
Guozhang
On Tue, Jan 4, 2022 at 6:31 PM Miguel González <mi...@klar.mx>
wrote:
> Hello
>
> Here's my store definition and my transformer with the whole logic (I
> included the transformer with issues and another version (V2) of the
> transformer with some improvements that I believe are necessary)
>
> https://gist.github.com/magg/576bf3381c9c0501b9761b54e9d86375
>
> Thanks
> - Miguel
>
>
>
>
>
> On Tue, Jan 4, 2022 at 5:21 PM Guozhang Wang <wa...@gmail.com> wrote:
>
> > Hi Miguel,
> >
> > How is your kvStore being constructed? Could you paste the snippet of the
> > related construction code, as well as the related iterating / deletion
> code
> > here?
> >
> > On Tue, Jan 4, 2022 at 2:25 PM Matthias J. Sax <mj...@apache.org> wrote:
> >
> > > Not 100% sure. From what you describe it should work as expected.
> > >
> > > It seems `delete()` does not delete the key from the store (ie,
> RocksDB)
> > > itself (for unknown reasons)?
> > >
> > > Are you closing all your iterators correctly? (More or less a wild
> guess
> > > at the moment.)
> > >
> > > Did you enable caching for the store? (Just to double check if it could
> > > be caching related or not.)
> > >
> > >
> > > -Matthias
> > >
> > >
> > > On 12/24/21 11:08 AM, Miguel González wrote:
> > > > Hello
> > > >
> > > > I'm using Kafka Streams and I have a transformer that uses
> > > > a TimestampedKeyValueStore, I have a punctuator that is in charge of
> > > > cleaning the store,
> > > >
> > > > Basically I'm iterating the store using kvStore.all() and deleting
> the
> > > keys
> > > > based on some logic with kvStore.delete(key);
> > > >
> > > > I'm seeing the changelog topic for the store grow unbounded, I'm
> seeing
> > > > many values with null for the same keys... I think those are called
> > > > tombstones right? but the punctuator is constantly doing the same
> > thing
> > > > trying to delete the same keys.. I see more tombstones being
> inserted.
> > > >
> > > > Is this the expected behavior? If so, how can I correctly clean that
> > > store?
> > > >
> > > > thanks
> > > > - Miguel
> > > >
> > >
> >
> >
> > --
> > -- Guozhang
> >
>
--
-- Guozhang
Re: How to properly use a clean a TimestampedKeyValueStore
Posted by Miguel González <mi...@klar.mx>.
Hello
Here's my store definition and my transformer with the whole logic (I
included the transformer with issues and another version (V2) of the
transformer with some improvements that I believe are necessary)
https://gist.github.com/magg/576bf3381c9c0501b9761b54e9d86375
Thanks
- Miguel
On Tue, Jan 4, 2022 at 5:21 PM Guozhang Wang <wa...@gmail.com> wrote:
> Hi Miguel,
>
> How is your kvStore being constructed? Could you paste the snippet of the
> related construction code, as well as the related iterating / deletion code
> here?
>
> On Tue, Jan 4, 2022 at 2:25 PM Matthias J. Sax <mj...@apache.org> wrote:
>
> > Not 100% sure. From what you describe it should work as expected.
> >
> > It seems `delete()` does not delete the key from the store (ie, RocksDB)
> > itself (for unknown reasons)?
> >
> > Are you closing all your iterators correctly? (More or less a wild guess
> > at the moment.)
> >
> > Did you enable caching for the store? (Just to double check if it could
> > be caching related or not.)
> >
> >
> > -Matthias
> >
> >
> > On 12/24/21 11:08 AM, Miguel González wrote:
> > > Hello
> > >
> > > I'm using Kafka Streams and I have a transformer that uses
> > > a TimestampedKeyValueStore, I have a punctuator that is in charge of
> > > cleaning the store,
> > >
> > > Basically I'm iterating the store using kvStore.all() and deleting the
> > keys
> > > based on some logic with kvStore.delete(key);
> > >
> > > I'm seeing the changelog topic for the store grow unbounded, I'm seeing
> > > many values with null for the same keys... I think those are called
> > > tombstones right? but the punctuator is constantly doing the same
> thing
> > > trying to delete the same keys.. I see more tombstones being inserted.
> > >
> > > Is this the expected behavior? If so, how can I correctly clean that
> > store?
> > >
> > > thanks
> > > - Miguel
> > >
> >
>
>
> --
> -- Guozhang
>
Re: How to properly use a clean a TimestampedKeyValueStore
Posted by Guozhang Wang <wa...@gmail.com>.
Hi Miguel,
How is your kvStore being constructed? Could you paste the snippet of the
related construction code, as well as the related iterating / deletion code
here?
On Tue, Jan 4, 2022 at 2:25 PM Matthias J. Sax <mj...@apache.org> wrote:
> Not 100% sure. From what you describe it should work as expected.
>
> It seems `delete()` does not delete the key from the store (ie, RocksDB)
> itself (for unknown reasons)?
>
> Are you closing all your iterators correctly? (More or less a wild guess
> at the moment.)
>
> Did you enable caching for the store? (Just to double check if it could
> be caching related or not.)
>
>
> -Matthias
>
>
> On 12/24/21 11:08 AM, Miguel González wrote:
> > Hello
> >
> > I'm using Kafka Streams and I have a transformer that uses
> > a TimestampedKeyValueStore, I have a punctuator that is in charge of
> > cleaning the store,
> >
> > Basically I'm iterating the store using kvStore.all() and deleting the
> keys
> > based on some logic with kvStore.delete(key);
> >
> > I'm seeing the changelog topic for the store grow unbounded, I'm seeing
> > many values with null for the same keys... I think those are called
> > tombstones right? but the punctuator is constantly doing the same thing
> > trying to delete the same keys.. I see more tombstones being inserted.
> >
> > Is this the expected behavior? If so, how can I correctly clean that
> store?
> >
> > thanks
> > - Miguel
> >
>
--
-- Guozhang
Re: How to properly use a clean a TimestampedKeyValueStore
Posted by "Matthias J. Sax" <mj...@apache.org>.
Not 100% sure. From what you describe it should work as expected.
It seems `delete()` does not delete the key from the store (ie, RocksDB)
itself (for unknown reasons)?
Are you closing all your iterators correctly? (More or less a wild guess
at the moment.)
Did you enable caching for the store? (Just to double check if it could
be caching related or not.)
-Matthias
On 12/24/21 11:08 AM, Miguel González wrote:
> Hello
>
> I'm using Kafka Streams and I have a transformer that uses
> a TimestampedKeyValueStore, I have a punctuator that is in charge of
> cleaning the store,
>
> Basically I'm iterating the store using kvStore.all() and deleting the keys
> based on some logic with kvStore.delete(key);
>
> I'm seeing the changelog topic for the store grow unbounded, I'm seeing
> many values with null for the same keys... I think those are called
> tombstones right? but the punctuator is constantly doing the same thing
> trying to delete the same keys.. I see more tombstones being inserted.
>
> Is this the expected behavior? If so, how can I correctly clean that store?
>
> thanks
> - Miguel
>