You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Evan Zhou via dev <de...@kafka.apache.org> on 2025/12/11 00:39:35 UTC
[DISCUSS] KIP-1250: Add metric to track size of in-memory state stores
Hi all,
I'd like to start the discussion for KIP-1250, which adds a new metric to
track the size of in-memory state stores. Today, a similar metric exists in
Kafka, but only for RocksDB, and this KIP intends to close that gap.
https://cwiki.apache.org/confluence/x/noTMFw
Thanks,
Evan
Re: [DISCUSS] KIP-1250: Add metric to track size of in-memory state stores
Posted by Evan Zhou via dev <de...@kafka.apache.org>.
Hi Bill,
Sounds good, I'll keep the metric name as is.
Thanks,
Evan
On Wed, Jan 14, 2026 at 3:08 PM Bill Bejeck <bb...@gmail.com> wrote:
> Thanks for the KIP Evan, this fills a gap in the metrics for state stores.
>
> With the naming, I agree with Matthias that we probably don't need to do
> anything specific and as it stands is good enough.
> I also think it should be available for all in-memory store types.
>
> Other than that, the KIP lgtm.
>
> Thanks,
> Bill
>
>
>
> On Mon, Jan 12, 2026 at 7:12 PM Matthias J. Sax <mj...@apache.org> wrote:
>
> > Thanks for the KIP Evan.
> >
> > MJS1: Why is the metric added at DEBUG level? The corresponding RocksDB
> > metric is added at INFO level. The KIP says:
> >
> > > The addition of this metric will have no effect on existing users. The
> > performance impact of adding this metric is non-trivial, thus we only
> > record it at the DEBUG level.
> >
> > Why is it expensive? Not clear to me atm. And why would it be cheaper
> > for the RocksDB case?
> >
> >
> >
> >
> > About Lucas question: Not sure if we would need to do anything about it?
> > Given that we have specific RocksDB metrics, which are not available for
> > in-memory stores, why should we not have specific in-memory store
> > metrics? RocksDB metrics are also not containing "rocksdb" in their
> names?
> >
> > As long as it's properly document as "in-memory metric" (and the
> > description already mentions it explicitly) it might just be fine?
> >
> >
> >
> > MJS2: The KIP says, we would implement this only for
> > `MeteredKeyValueStore`. Why not also add it to the other store types
> > like windowed and others?
> >
> >
> > -Matthias
> >
> >
> > On 12/12/25 4:14 PM, Evan Zhou via dev wrote:
> > > Hi Lucas,
> > >
> > > Thanks for the feedback.
> > >
> > > My original intention was to use "estimate" in the metric name as the
> > > differentiator, but I agree that this is not very clear. How does
> > changing
> > > the metric name to "in-memory-state-num-keys" sound?
> > >
> > > Thanks,
> > > Evan
> > >
> > > On Thu, Dec 11, 2025 at 1:21 AM Lucas Brutschy via dev <
> > dev@kafka.apache.org>
> > > wrote:
> > >
> > >> Hi Evan,
> > >>
> > >> thanks for the KIP!
> > >>
> > >> If this is specific to in-memory stores, I wonder if we should add
> > >> this to the metric name? People could become confused that rocksDB
> > >> state is not showing up.
> > >>
> > >> Cheers,
> > >> Lucas
> > >>
> > >> On Thu, Dec 11, 2025 at 1:40 AM Evan Zhou via dev <
> dev@kafka.apache.org
> > >
> > >> wrote:
> > >>>
> > >>> Hi all,
> > >>>
> > >>> I'd like to start the discussion for KIP-1250, which adds a new
> metric
> > to
> > >>> track the size of in-memory state stores. Today, a similar metric
> > exists
> > >> in
> > >>> Kafka, but only for RocksDB, and this KIP intends to close that gap.
> > >>>
> > >>> https://cwiki.apache.org/confluence/x/noTMFw
> > >>>
> > >>> Thanks,
> > >>> Evan
> > >>
> > >
> >
> >
>
Re: [DISCUSS] KIP-1250: Add metric to track size of in-memory state stores
Posted by Bill Bejeck <bb...@gmail.com>.
Thanks for the KIP Evan, this fills a gap in the metrics for state stores.
With the naming, I agree with Matthias that we probably don't need to do
anything specific and as it stands is good enough.
I also think it should be available for all in-memory store types.
Other than that, the KIP lgtm.
Thanks,
Bill
On Mon, Jan 12, 2026 at 7:12 PM Matthias J. Sax <mj...@apache.org> wrote:
> Thanks for the KIP Evan.
>
> MJS1: Why is the metric added at DEBUG level? The corresponding RocksDB
> metric is added at INFO level. The KIP says:
>
> > The addition of this metric will have no effect on existing users. The
> performance impact of adding this metric is non-trivial, thus we only
> record it at the DEBUG level.
>
> Why is it expensive? Not clear to me atm. And why would it be cheaper
> for the RocksDB case?
>
>
>
>
> About Lucas question: Not sure if we would need to do anything about it?
> Given that we have specific RocksDB metrics, which are not available for
> in-memory stores, why should we not have specific in-memory store
> metrics? RocksDB metrics are also not containing "rocksdb" in their names?
>
> As long as it's properly document as "in-memory metric" (and the
> description already mentions it explicitly) it might just be fine?
>
>
>
> MJS2: The KIP says, we would implement this only for
> `MeteredKeyValueStore`. Why not also add it to the other store types
> like windowed and others?
>
>
> -Matthias
>
>
> On 12/12/25 4:14 PM, Evan Zhou via dev wrote:
> > Hi Lucas,
> >
> > Thanks for the feedback.
> >
> > My original intention was to use "estimate" in the metric name as the
> > differentiator, but I agree that this is not very clear. How does
> changing
> > the metric name to "in-memory-state-num-keys" sound?
> >
> > Thanks,
> > Evan
> >
> > On Thu, Dec 11, 2025 at 1:21 AM Lucas Brutschy via dev <
> dev@kafka.apache.org>
> > wrote:
> >
> >> Hi Evan,
> >>
> >> thanks for the KIP!
> >>
> >> If this is specific to in-memory stores, I wonder if we should add
> >> this to the metric name? People could become confused that rocksDB
> >> state is not showing up.
> >>
> >> Cheers,
> >> Lucas
> >>
> >> On Thu, Dec 11, 2025 at 1:40 AM Evan Zhou via dev <dev@kafka.apache.org
> >
> >> wrote:
> >>>
> >>> Hi all,
> >>>
> >>> I'd like to start the discussion for KIP-1250, which adds a new metric
> to
> >>> track the size of in-memory state stores. Today, a similar metric
> exists
> >> in
> >>> Kafka, but only for RocksDB, and this KIP intends to close that gap.
> >>>
> >>> https://cwiki.apache.org/confluence/x/noTMFw
> >>>
> >>> Thanks,
> >>> Evan
> >>
> >
>
>
Re: [DISCUSS] KIP-1250: Add metric to track size of in-memory state stores
Posted by Lucas Brutschy via dev <de...@kafka.apache.org>.
Agreed, let's vote on this.
Cheers,
Lucas
On Fri, Jan 23, 2026 at 3:18 AM Matthias J. Sax <mj...@apache.org> wrote:
>
> Thanks for the updates Evan. Sounds good. I did not look at the KIP
> again yet, but I think you can start a VOTE?
>
> -Matthias
>
> On 1/21/26 3:25 PM, Evan Zhou via dev wrote:
> > Hi Matthias,
> >
> > MJS1: I did some more digging, and I see what you were initially asking.
> > I've updated the KIP to specify that a gauge will be used to collect the
> > metric rather than a sensor, which will make the performance impact trivial
> > compared to the initial proposal. I've also changed the recording level to
> > be INFO, matching the behavior of the RocksDB equivalent metric.
> >
> > Thanks,
> > Evan
> >
> > On Fri, Jan 16, 2026 at 1:43 PM Evan Zhou <ez...@confluent.io> wrote:
> >
> >> Hi Matthias,
> >>
> >> MJS1: When trying to figure out how to record this metric, I figured the
> >> simplest and most straightforward way is to update the metric on every put
> >> and delete, which would have a nonzero performance impact, thus I decided
> >> to add it only at the DEBUG level. I'm not sure how RocksDB collects its
> >> equivalent metric, but my understanding is that Kafka Streams just passes
> >> up the value it gets from RocksDB and doesn't do any of the collecting
> >> itself, which is why I believe the in-memory implementation would be more
> >> expensive
> >>
> >> MJS2: Good point, I've updated the KIP to reflect this. To repeat what is
> >> in the KIP, this metric will be implemented for all classes starting with
> >> `Metered` under `streams/state/internals`. For completeness, the metered
> >> iterators are able to remove elements, thus they are included here.
> >>
> >> Thanks,
> >> Evan
> >>
> >> On Mon, Jan 12, 2026 at 6:13 PM Matthias J. Sax <mj...@apache.org> wrote:
> >>
> >>> Thanks for the KIP Evan.
> >>>
> >>> MJS1: Why is the metric added at DEBUG level? The corresponding RocksDB
> >>> metric is added at INFO level. The KIP says:
> >>>
> >>>> The addition of this metric will have no effect on existing users. The
> >>> performance impact of adding this metric is non-trivial, thus we only
> >>> record it at the DEBUG level.
> >>>
> >>> Why is it expensive? Not clear to me atm. And why would it be cheaper
> >>> for the RocksDB case?
> >>>
> >>>
> >>>
> >>>
> >>> About Lucas question: Not sure if we would need to do anything about it?
> >>> Given that we have specific RocksDB metrics, which are not available for
> >>> in-memory stores, why should we not have specific in-memory store
> >>> metrics? RocksDB metrics are also not containing "rocksdb" in their names?
> >>>
> >>> As long as it's properly document as "in-memory metric" (and the
> >>> description already mentions it explicitly) it might just be fine?
> >>>
> >>>
> >>>
> >>> MJS2: The KIP says, we would implement this only for
> >>> `MeteredKeyValueStore`. Why not also add it to the other store types
> >>> like windowed and others?
> >>>
> >>>
> >>> -Matthias
> >>>
> >>>
> >>> On 12/12/25 4:14 PM, Evan Zhou via dev wrote:
> >>>> Hi Lucas,
> >>>>
> >>>> Thanks for the feedback.
> >>>>
> >>>> My original intention was to use "estimate" in the metric name as the
> >>>> differentiator, but I agree that this is not very clear. How does
> >>> changing
> >>>> the metric name to "in-memory-state-num-keys" sound?
> >>>>
> >>>> Thanks,
> >>>> Evan
> >>>>
> >>>> On Thu, Dec 11, 2025 at 1:21 AM Lucas Brutschy via dev <
> >>> dev@kafka.apache.org>
> >>>> wrote:
> >>>>
> >>>>> Hi Evan,
> >>>>>
> >>>>> thanks for the KIP!
> >>>>>
> >>>>> If this is specific to in-memory stores, I wonder if we should add
> >>>>> this to the metric name? People could become confused that rocksDB
> >>>>> state is not showing up.
> >>>>>
> >>>>> Cheers,
> >>>>> Lucas
> >>>>>
> >>>>> On Thu, Dec 11, 2025 at 1:40 AM Evan Zhou via dev <
> >>> dev@kafka.apache.org>
> >>>>> wrote:
> >>>>>>
> >>>>>> Hi all,
> >>>>>>
> >>>>>> I'd like to start the discussion for KIP-1250, which adds a new
> >>> metric to
> >>>>>> track the size of in-memory state stores. Today, a similar metric
> >>> exists
> >>>>> in
> >>>>>> Kafka, but only for RocksDB, and this KIP intends to close that gap.
> >>>>>>
> >>>>>> https://cwiki.apache.org/confluence/x/noTMFw
> >>>>>>
> >>>>>> Thanks,
> >>>>>> Evan
> >>>>>
> >>>>
> >>>
> >>>
> >
>
Re: [DISCUSS] KIP-1250: Add metric to track size of in-memory state stores
Posted by "Matthias J. Sax" <mj...@apache.org>.
Thanks for the updates Evan. Sounds good. I did not look at the KIP
again yet, but I think you can start a VOTE?
-Matthias
On 1/21/26 3:25 PM, Evan Zhou via dev wrote:
> Hi Matthias,
>
> MJS1: I did some more digging, and I see what you were initially asking.
> I've updated the KIP to specify that a gauge will be used to collect the
> metric rather than a sensor, which will make the performance impact trivial
> compared to the initial proposal. I've also changed the recording level to
> be INFO, matching the behavior of the RocksDB equivalent metric.
>
> Thanks,
> Evan
>
> On Fri, Jan 16, 2026 at 1:43 PM Evan Zhou <ez...@confluent.io> wrote:
>
>> Hi Matthias,
>>
>> MJS1: When trying to figure out how to record this metric, I figured the
>> simplest and most straightforward way is to update the metric on every put
>> and delete, which would have a nonzero performance impact, thus I decided
>> to add it only at the DEBUG level. I'm not sure how RocksDB collects its
>> equivalent metric, but my understanding is that Kafka Streams just passes
>> up the value it gets from RocksDB and doesn't do any of the collecting
>> itself, which is why I believe the in-memory implementation would be more
>> expensive
>>
>> MJS2: Good point, I've updated the KIP to reflect this. To repeat what is
>> in the KIP, this metric will be implemented for all classes starting with
>> `Metered` under `streams/state/internals`. For completeness, the metered
>> iterators are able to remove elements, thus they are included here.
>>
>> Thanks,
>> Evan
>>
>> On Mon, Jan 12, 2026 at 6:13 PM Matthias J. Sax <mj...@apache.org> wrote:
>>
>>> Thanks for the KIP Evan.
>>>
>>> MJS1: Why is the metric added at DEBUG level? The corresponding RocksDB
>>> metric is added at INFO level. The KIP says:
>>>
>>>> The addition of this metric will have no effect on existing users. The
>>> performance impact of adding this metric is non-trivial, thus we only
>>> record it at the DEBUG level.
>>>
>>> Why is it expensive? Not clear to me atm. And why would it be cheaper
>>> for the RocksDB case?
>>>
>>>
>>>
>>>
>>> About Lucas question: Not sure if we would need to do anything about it?
>>> Given that we have specific RocksDB metrics, which are not available for
>>> in-memory stores, why should we not have specific in-memory store
>>> metrics? RocksDB metrics are also not containing "rocksdb" in their names?
>>>
>>> As long as it's properly document as "in-memory metric" (and the
>>> description already mentions it explicitly) it might just be fine?
>>>
>>>
>>>
>>> MJS2: The KIP says, we would implement this only for
>>> `MeteredKeyValueStore`. Why not also add it to the other store types
>>> like windowed and others?
>>>
>>>
>>> -Matthias
>>>
>>>
>>> On 12/12/25 4:14 PM, Evan Zhou via dev wrote:
>>>> Hi Lucas,
>>>>
>>>> Thanks for the feedback.
>>>>
>>>> My original intention was to use "estimate" in the metric name as the
>>>> differentiator, but I agree that this is not very clear. How does
>>> changing
>>>> the metric name to "in-memory-state-num-keys" sound?
>>>>
>>>> Thanks,
>>>> Evan
>>>>
>>>> On Thu, Dec 11, 2025 at 1:21 AM Lucas Brutschy via dev <
>>> dev@kafka.apache.org>
>>>> wrote:
>>>>
>>>>> Hi Evan,
>>>>>
>>>>> thanks for the KIP!
>>>>>
>>>>> If this is specific to in-memory stores, I wonder if we should add
>>>>> this to the metric name? People could become confused that rocksDB
>>>>> state is not showing up.
>>>>>
>>>>> Cheers,
>>>>> Lucas
>>>>>
>>>>> On Thu, Dec 11, 2025 at 1:40 AM Evan Zhou via dev <
>>> dev@kafka.apache.org>
>>>>> wrote:
>>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> I'd like to start the discussion for KIP-1250, which adds a new
>>> metric to
>>>>>> track the size of in-memory state stores. Today, a similar metric
>>> exists
>>>>> in
>>>>>> Kafka, but only for RocksDB, and this KIP intends to close that gap.
>>>>>>
>>>>>> https://cwiki.apache.org/confluence/x/noTMFw
>>>>>>
>>>>>> Thanks,
>>>>>> Evan
>>>>>
>>>>
>>>
>>>
>
Re: [DISCUSS] KIP-1250: Add metric to track size of in-memory state stores
Posted by Evan Zhou via dev <de...@kafka.apache.org>.
Hi Matthias,
MJS1: I did some more digging, and I see what you were initially asking.
I've updated the KIP to specify that a gauge will be used to collect the
metric rather than a sensor, which will make the performance impact trivial
compared to the initial proposal. I've also changed the recording level to
be INFO, matching the behavior of the RocksDB equivalent metric.
Thanks,
Evan
On Fri, Jan 16, 2026 at 1:43 PM Evan Zhou <ez...@confluent.io> wrote:
> Hi Matthias,
>
> MJS1: When trying to figure out how to record this metric, I figured the
> simplest and most straightforward way is to update the metric on every put
> and delete, which would have a nonzero performance impact, thus I decided
> to add it only at the DEBUG level. I'm not sure how RocksDB collects its
> equivalent metric, but my understanding is that Kafka Streams just passes
> up the value it gets from RocksDB and doesn't do any of the collecting
> itself, which is why I believe the in-memory implementation would be more
> expensive
>
> MJS2: Good point, I've updated the KIP to reflect this. To repeat what is
> in the KIP, this metric will be implemented for all classes starting with
> `Metered` under `streams/state/internals`. For completeness, the metered
> iterators are able to remove elements, thus they are included here.
>
> Thanks,
> Evan
>
> On Mon, Jan 12, 2026 at 6:13 PM Matthias J. Sax <mj...@apache.org> wrote:
>
>> Thanks for the KIP Evan.
>>
>> MJS1: Why is the metric added at DEBUG level? The corresponding RocksDB
>> metric is added at INFO level. The KIP says:
>>
>> > The addition of this metric will have no effect on existing users. The
>> performance impact of adding this metric is non-trivial, thus we only
>> record it at the DEBUG level.
>>
>> Why is it expensive? Not clear to me atm. And why would it be cheaper
>> for the RocksDB case?
>>
>>
>>
>>
>> About Lucas question: Not sure if we would need to do anything about it?
>> Given that we have specific RocksDB metrics, which are not available for
>> in-memory stores, why should we not have specific in-memory store
>> metrics? RocksDB metrics are also not containing "rocksdb" in their names?
>>
>> As long as it's properly document as "in-memory metric" (and the
>> description already mentions it explicitly) it might just be fine?
>>
>>
>>
>> MJS2: The KIP says, we would implement this only for
>> `MeteredKeyValueStore`. Why not also add it to the other store types
>> like windowed and others?
>>
>>
>> -Matthias
>>
>>
>> On 12/12/25 4:14 PM, Evan Zhou via dev wrote:
>> > Hi Lucas,
>> >
>> > Thanks for the feedback.
>> >
>> > My original intention was to use "estimate" in the metric name as the
>> > differentiator, but I agree that this is not very clear. How does
>> changing
>> > the metric name to "in-memory-state-num-keys" sound?
>> >
>> > Thanks,
>> > Evan
>> >
>> > On Thu, Dec 11, 2025 at 1:21 AM Lucas Brutschy via dev <
>> dev@kafka.apache.org>
>> > wrote:
>> >
>> >> Hi Evan,
>> >>
>> >> thanks for the KIP!
>> >>
>> >> If this is specific to in-memory stores, I wonder if we should add
>> >> this to the metric name? People could become confused that rocksDB
>> >> state is not showing up.
>> >>
>> >> Cheers,
>> >> Lucas
>> >>
>> >> On Thu, Dec 11, 2025 at 1:40 AM Evan Zhou via dev <
>> dev@kafka.apache.org>
>> >> wrote:
>> >>>
>> >>> Hi all,
>> >>>
>> >>> I'd like to start the discussion for KIP-1250, which adds a new
>> metric to
>> >>> track the size of in-memory state stores. Today, a similar metric
>> exists
>> >> in
>> >>> Kafka, but only for RocksDB, and this KIP intends to close that gap.
>> >>>
>> >>> https://cwiki.apache.org/confluence/x/noTMFw
>> >>>
>> >>> Thanks,
>> >>> Evan
>> >>
>> >
>>
>>
Re: [DISCUSS] KIP-1250: Add metric to track size of in-memory state stores
Posted by Evan Zhou via dev <de...@kafka.apache.org>.
Hi Matthias,
MJS1: When trying to figure out how to record this metric, I figured the
simplest and most straightforward way is to update the metric on every put
and delete, which would have a nonzero performance impact, thus I decided
to add it only at the DEBUG level. I'm not sure how RocksDB collects its
equivalent metric, but my understanding is that Kafka Streams just passes
up the value it gets from RocksDB and doesn't do any of the collecting
itself, which is why I believe the in-memory implementation would be more
expensive
MJS2: Good point, I've updated the KIP to reflect this. To repeat what is
in the KIP, this metric will be implemented for all classes starting with
`Metered` under `streams/state/internals`. For completeness, the metered
iterators are able to remove elements, thus they are included here.
Thanks,
Evan
On Mon, Jan 12, 2026 at 6:13 PM Matthias J. Sax <mj...@apache.org> wrote:
> Thanks for the KIP Evan.
>
> MJS1: Why is the metric added at DEBUG level? The corresponding RocksDB
> metric is added at INFO level. The KIP says:
>
> > The addition of this metric will have no effect on existing users. The
> performance impact of adding this metric is non-trivial, thus we only
> record it at the DEBUG level.
>
> Why is it expensive? Not clear to me atm. And why would it be cheaper
> for the RocksDB case?
>
>
>
>
> About Lucas question: Not sure if we would need to do anything about it?
> Given that we have specific RocksDB metrics, which are not available for
> in-memory stores, why should we not have specific in-memory store
> metrics? RocksDB metrics are also not containing "rocksdb" in their names?
>
> As long as it's properly document as "in-memory metric" (and the
> description already mentions it explicitly) it might just be fine?
>
>
>
> MJS2: The KIP says, we would implement this only for
> `MeteredKeyValueStore`. Why not also add it to the other store types
> like windowed and others?
>
>
> -Matthias
>
>
> On 12/12/25 4:14 PM, Evan Zhou via dev wrote:
> > Hi Lucas,
> >
> > Thanks for the feedback.
> >
> > My original intention was to use "estimate" in the metric name as the
> > differentiator, but I agree that this is not very clear. How does
> changing
> > the metric name to "in-memory-state-num-keys" sound?
> >
> > Thanks,
> > Evan
> >
> > On Thu, Dec 11, 2025 at 1:21 AM Lucas Brutschy via dev <
> dev@kafka.apache.org>
> > wrote:
> >
> >> Hi Evan,
> >>
> >> thanks for the KIP!
> >>
> >> If this is specific to in-memory stores, I wonder if we should add
> >> this to the metric name? People could become confused that rocksDB
> >> state is not showing up.
> >>
> >> Cheers,
> >> Lucas
> >>
> >> On Thu, Dec 11, 2025 at 1:40 AM Evan Zhou via dev <dev@kafka.apache.org
> >
> >> wrote:
> >>>
> >>> Hi all,
> >>>
> >>> I'd like to start the discussion for KIP-1250, which adds a new metric
> to
> >>> track the size of in-memory state stores. Today, a similar metric
> exists
> >> in
> >>> Kafka, but only for RocksDB, and this KIP intends to close that gap.
> >>>
> >>> https://cwiki.apache.org/confluence/x/noTMFw
> >>>
> >>> Thanks,
> >>> Evan
> >>
> >
>
>
Re: [DISCUSS] KIP-1250: Add metric to track size of in-memory state stores
Posted by "Matthias J. Sax" <mj...@apache.org>.
Thanks for the KIP Evan.
MJS1: Why is the metric added at DEBUG level? The corresponding RocksDB
metric is added at INFO level. The KIP says:
> The addition of this metric will have no effect on existing users. The performance impact of adding this metric is non-trivial, thus we only record it at the DEBUG level.
Why is it expensive? Not clear to me atm. And why would it be cheaper
for the RocksDB case?
About Lucas question: Not sure if we would need to do anything about it?
Given that we have specific RocksDB metrics, which are not available for
in-memory stores, why should we not have specific in-memory store
metrics? RocksDB metrics are also not containing "rocksdb" in their names?
As long as it's properly document as "in-memory metric" (and the
description already mentions it explicitly) it might just be fine?
MJS2: The KIP says, we would implement this only for
`MeteredKeyValueStore`. Why not also add it to the other store types
like windowed and others?
-Matthias
On 12/12/25 4:14 PM, Evan Zhou via dev wrote:
> Hi Lucas,
>
> Thanks for the feedback.
>
> My original intention was to use "estimate" in the metric name as the
> differentiator, but I agree that this is not very clear. How does changing
> the metric name to "in-memory-state-num-keys" sound?
>
> Thanks,
> Evan
>
> On Thu, Dec 11, 2025 at 1:21 AM Lucas Brutschy via dev <de...@kafka.apache.org>
> wrote:
>
>> Hi Evan,
>>
>> thanks for the KIP!
>>
>> If this is specific to in-memory stores, I wonder if we should add
>> this to the metric name? People could become confused that rocksDB
>> state is not showing up.
>>
>> Cheers,
>> Lucas
>>
>> On Thu, Dec 11, 2025 at 1:40 AM Evan Zhou via dev <de...@kafka.apache.org>
>> wrote:
>>>
>>> Hi all,
>>>
>>> I'd like to start the discussion for KIP-1250, which adds a new metric to
>>> track the size of in-memory state stores. Today, a similar metric exists
>> in
>>> Kafka, but only for RocksDB, and this KIP intends to close that gap.
>>>
>>> https://cwiki.apache.org/confluence/x/noTMFw
>>>
>>> Thanks,
>>> Evan
>>
>
Re: [DISCUSS] KIP-1250: Add metric to track size of in-memory state stores
Posted by Evan Zhou via dev <de...@kafka.apache.org>.
Hi Lucas,
Thanks for the feedback.
My original intention was to use "estimate" in the metric name as the
differentiator, but I agree that this is not very clear. How does changing
the metric name to "in-memory-state-num-keys" sound?
Thanks,
Evan
On Thu, Dec 11, 2025 at 1:21 AM Lucas Brutschy via dev <de...@kafka.apache.org>
wrote:
> Hi Evan,
>
> thanks for the KIP!
>
> If this is specific to in-memory stores, I wonder if we should add
> this to the metric name? People could become confused that rocksDB
> state is not showing up.
>
> Cheers,
> Lucas
>
> On Thu, Dec 11, 2025 at 1:40 AM Evan Zhou via dev <de...@kafka.apache.org>
> wrote:
> >
> > Hi all,
> >
> > I'd like to start the discussion for KIP-1250, which adds a new metric to
> > track the size of in-memory state stores. Today, a similar metric exists
> in
> > Kafka, but only for RocksDB, and this KIP intends to close that gap.
> >
> > https://cwiki.apache.org/confluence/x/noTMFw
> >
> > Thanks,
> > Evan
>
Re: [DISCUSS] KIP-1250: Add metric to track size of in-memory state stores
Posted by Lucas Brutschy via dev <de...@kafka.apache.org>.
Hi Evan,
thanks for the KIP!
If this is specific to in-memory stores, I wonder if we should add
this to the metric name? People could become confused that rocksDB
state is not showing up.
Cheers,
Lucas
On Thu, Dec 11, 2025 at 1:40 AM Evan Zhou via dev <de...@kafka.apache.org> wrote:
>
> Hi all,
>
> I'd like to start the discussion for KIP-1250, which adds a new metric to
> track the size of in-memory state stores. Today, a similar metric exists in
> Kafka, but only for RocksDB, and this KIP intends to close that gap.
>
> https://cwiki.apache.org/confluence/x/noTMFw
>
> Thanks,
> Evan