You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Patrik Kleindl <pk...@gmail.com> on 2018/10/25 09:50:39 UTC
Stream Metrics - Memory Analysis
Hello
During the analysis of JVM memory two possible issues were shown which I
would like to bring to your attention:
1) Duplicate strings
Top findings:
string_content="stream-processor-node-metrics" count="534,277"
string_content="processor-node-id" count="148,437"
string_content="stream-rocksdb-state-metrics" count="41,832"
string_content="punctuate-latency-avg" count="29,681"
"stream-processor-node-metrics" seems to be used in Sensors.java as a
literal and not interned.
2) The HashMap parentSensors
from org.apache.kafka.streams.processor.internals.StreamThread$StreamsMetricsThreadImpl
was reported multiple times as suspicious for potentially keeping alive a
lot of objects. In our case the reported size was 40-50MB each.
I haven't looked too deep in the code but noticed that the class
Sensor.java which is used as a key in the HashMap does not implement equals
or hashCode method. Not sure this is a problem though.
Maybe someone can shed some light on this
best regards
Patrik
Re: Stream Metrics - Memory Analysis
Posted by Guozhang Wang <wa...@gmail.com>.
Thank you!
On Tue, Nov 20, 2018 at 3:44 AM Patrik Kleindl <pk...@gmail.com> wrote:
> Done.
> https://issues.apache.org/jira/browse/KAFKA-7660
> br, Patrik
>
> On Mon, 19 Nov 2018 at 02:03, Guozhang Wang <wa...@gmail.com> wrote:
>
> > Hello Patrik,
> >
> > Could you file a JIRA for your findings? Also what Kafka versions are you
> > using (could you add that to the ticket as well)?
> >
> > Could you provide some more elaborations on what you did the JVM
> analysis,
> > so that I can try to re-produce the observations.
> >
> >
> > Guozhang
> >
> > On Thu, Oct 25, 2018 at 2:50 AM Patrik Kleindl <pk...@gmail.com>
> wrote:
> >
> > > Hello
> > >
> > > During the analysis of JVM memory two possible issues were shown which
> I
> > > would like to bring to your attention:
> > > 1) Duplicate strings
> > > Top findings:
> > > string_content="stream-processor-node-metrics" count="534,277"
> > > string_content="processor-node-id" count="148,437"
> > > string_content="stream-rocksdb-state-metrics" count="41,832"
> > > string_content="punctuate-latency-avg" count="29,681"
> > >
> > > "stream-processor-node-metrics" seems to be used in Sensors.java as a
> > > literal and not interned.
> > >
> > > 2) The HashMap parentSensors
> > > from
> > >
> >
> org.apache.kafka.streams.processor.internals.StreamThread$StreamsMetricsThreadImpl
> > > was reported multiple times as suspicious for potentially keeping
> alive a
> > > lot of objects. In our case the reported size was 40-50MB each.
> > > I haven't looked too deep in the code but noticed that the class
> > > Sensor.java which is used as a key in the HashMap does not implement
> > equals
> > > or hashCode method. Not sure this is a problem though.
> > >
> > > Maybe someone can shed some light on this
> > >
> > > best regards
> > >
> > > Patrik
> > >
> >
> >
> > --
> > -- Guozhang
> >
>
--
-- Guozhang
Re: Stream Metrics - Memory Analysis
Posted by Patrik Kleindl <pk...@gmail.com>.
Done.
https://issues.apache.org/jira/browse/KAFKA-7660
br, Patrik
On Mon, 19 Nov 2018 at 02:03, Guozhang Wang <wa...@gmail.com> wrote:
> Hello Patrik,
>
> Could you file a JIRA for your findings? Also what Kafka versions are you
> using (could you add that to the ticket as well)?
>
> Could you provide some more elaborations on what you did the JVM analysis,
> so that I can try to re-produce the observations.
>
>
> Guozhang
>
> On Thu, Oct 25, 2018 at 2:50 AM Patrik Kleindl <pk...@gmail.com> wrote:
>
> > Hello
> >
> > During the analysis of JVM memory two possible issues were shown which I
> > would like to bring to your attention:
> > 1) Duplicate strings
> > Top findings:
> > string_content="stream-processor-node-metrics" count="534,277"
> > string_content="processor-node-id" count="148,437"
> > string_content="stream-rocksdb-state-metrics" count="41,832"
> > string_content="punctuate-latency-avg" count="29,681"
> >
> > "stream-processor-node-metrics" seems to be used in Sensors.java as a
> > literal and not interned.
> >
> > 2) The HashMap parentSensors
> > from
> >
> org.apache.kafka.streams.processor.internals.StreamThread$StreamsMetricsThreadImpl
> > was reported multiple times as suspicious for potentially keeping alive a
> > lot of objects. In our case the reported size was 40-50MB each.
> > I haven't looked too deep in the code but noticed that the class
> > Sensor.java which is used as a key in the HashMap does not implement
> equals
> > or hashCode method. Not sure this is a problem though.
> >
> > Maybe someone can shed some light on this
> >
> > best regards
> >
> > Patrik
> >
>
>
> --
> -- Guozhang
>
Re: Stream Metrics - Memory Analysis
Posted by Guozhang Wang <wa...@gmail.com>.
Hello Patrik,
Could you file a JIRA for your findings? Also what Kafka versions are you
using (could you add that to the ticket as well)?
Could you provide some more elaborations on what you did the JVM analysis,
so that I can try to re-produce the observations.
Guozhang
On Thu, Oct 25, 2018 at 2:50 AM Patrik Kleindl <pk...@gmail.com> wrote:
> Hello
>
> During the analysis of JVM memory two possible issues were shown which I
> would like to bring to your attention:
> 1) Duplicate strings
> Top findings:
> string_content="stream-processor-node-metrics" count="534,277"
> string_content="processor-node-id" count="148,437"
> string_content="stream-rocksdb-state-metrics" count="41,832"
> string_content="punctuate-latency-avg" count="29,681"
>
> "stream-processor-node-metrics" seems to be used in Sensors.java as a
> literal and not interned.
>
> 2) The HashMap parentSensors
> from
> org.apache.kafka.streams.processor.internals.StreamThread$StreamsMetricsThreadImpl
> was reported multiple times as suspicious for potentially keeping alive a
> lot of objects. In our case the reported size was 40-50MB each.
> I haven't looked too deep in the code but noticed that the class
> Sensor.java which is used as a key in the HashMap does not implement equals
> or hashCode method. Not sure this is a problem though.
>
> Maybe someone can shed some light on this
>
> best regards
>
> Patrik
>
--
-- Guozhang