You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Pushkar Deole <pd...@gmail.com> on 2020/07/14 09:57:16 UTC
kafka per topic metrics
Hi All,
Need some help on kafka metrics, i am interested in certain metrics e.g. i
need to know the number of records published on a particular topic and
number of records consumed from that topic by a specific consumer group, i
would need a total of these 2 and also average per second for them.
Are those metrics available on kafka brokers and if yes then which are
those metrics that would give me above counts?
Re: kafka per topic metrics
Posted by Pushkar Deole <pd...@gmail.com>.
Thanks Liam... went through the book and got many of doubts cleared..
I think apart from BrokerTopicsMetrics available at broker, there are many
other per topic metrics available at consumer and producers. Are you also
fetching those client app metrics through jmx exporter?
On Thu, Jul 16, 2020 at 11:16 AM Liam Clarke-Hutchinson <
liam.clarke@adscale.co.nz> wrote:
> Hi Pushkar,
>
> Best way to see what metrics are available is to connect to a broker via
> JConsole to see the exposed mbeans.
>
> You can iterate over them programmatically by using the MBean API.
>
> Also recommend chapter 10 of Kafka: The Definitive Guide, it covers the
> metrics really well.
>
> Cheers,
>
> Liam Clarke-Hutchinson
>
> On Thu, 16 Jul. 2020, 5:05 pm Pushkar Deole, <pd...@gmail.com> wrote:
>
> > Thanks Liam...
> > Few questions: in your pattern the topic parameter is appended pattern:
> > 'kafka.server<type=BrokerTopicMetrics, name=(.+),
> > topic=(.+)><>OneMinuteRate'
> > however the kafka docs doesn't mention that
> > kafka.server:type=BrokerTopicMetrics,name=MessagesInPerSec
> >
> > does the topic parameter available in all BrokerTopicMetrics and can
> the
> > broker provide that parameter dynamically by iterating through all topics
> > or selective topics? Where will the logic reside to iterate through
> > available topics and export metrics for all of them?
> >
> > On Thu, Jul 16, 2020 at 10:04 AM Liam Clarke-Hutchinson <
> > liam.clarke@adscale.co.nz> wrote:
> >
> > > Whoops, just spotted a typo - the second $1 in the above snippet should
> > of
> > > course be $2.
> > >
> > > On Thu, Jul 16, 2020 at 4:33 PM Liam Clarke-Hutchinson <
> > > liam.clarke@adscale.co.nz> wrote:
> > >
> > > > Hi Pushkar,
> > > >
> > > > There are broker side metrics for messages in / bytes in / bytes out
> > per
> > > > topic per second. I use this jmx_exporter rule to export them:
> > > > - pattern: 'kafka.server<type=BrokerTopicMetrics, name=(.+),
> > > > topic=(.+)><>OneMinuteRate'
> > > > name: kafka_broker_per_topic_$1_one_minute_rate
> > > > labels:
> > > > topic: $1
> > > > type: GAUGE
> > > >
> > > > You can't get the number of messages out per topic from the broker
> > > > because... I think it's somehow related to batching, or similar, it
> > > doesn't
> > > > count messages out, only bytes out. You can, however, get that metric
> > > from
> > > > the consumer if you're using the Java Kafka client, there's a
> per-topic
> > > > messages consumed per second metric exposed as an MBean.
> > > >
> > > > You could either use jmx_exporter to also export that from the client
> > > app,
> > > > or if possible, add some code that connects to the mbean inside the
> JVM
> > > and
> > > > then exports it via any pre-existing Prometheus registry. You might
> > want
> > > to
> > > > then use a Prometheus aggregating rule to collate all the
> per-consumer
> > > apps
> > > > into a per-consumer-group metric, unless the per consumer granularity
> > is
> > > of
> > > > interest to you: https://prometheus.io/docs/practices/rules/
> > > >
> > > > Hope that helps,
> > > >
> > > > Kind regards,
> > > >
> > > > Liam Clarke-Hutchinsons
> > > >
> > > > On Thu, 16 Jul. 2020, 3:46 pm Pushkar Deole, <pd...@gmail.com>
> > > wrote:
> > > >
> > > >> Thanks Claudia! For broker level metrics, we are also using same jmx
> > > >> exporter to export those metrics to prometheus.
> > > >> Are you fetching any per topic metrics from broker? e.g. messages
> > > produced
> > > >> on a certain topic or messages consumed from a certain topic. I am
> > > mainly
> > > >> interested in these metrics.
> > > >>
> > > >> I read in kafka docs that they are present at producer/consumer,
> > > however I
> > > >> am not sure how to fetch them from consumer/producer.
> > > >>
> > > >> On Wed, Jul 15, 2020 at 8:32 PM Claudia Kesslau <
> c.kesslau@kasasi.de>
> > > >> wrote:
> > > >>
> > > >> > Hi,
> > > >> >
> > > >> > I use https://github.com/prometheus/jmx_exporter for collecting
> > > broker
> > > >> > metrics and integrating them into prometheus.
> > > >> >
> > > >> > Hope this helps.
> > > >> > Greetings,
> > > >> > Claudia
> > > >> >
> > > >> > -----Ursprüngliche Nachricht-----
> > > >> > Von: Pushkar Deole <pd...@gmail.com>
> > > >> > Gesendet: Mittwoch, 15. Juli 2020 09:07
> > > >> > An: users@kafka.apache.org
> > > >> > Betreff: Re: kafka per topic metrics
> > > >> >
> > > >> > We are using prometheus as metrics collection and storage system
> and
> > > >> > Grafana for displaying those metrics, so integration with them is
> > > >> required
> > > >> >
> > > >> > On Wed, Jul 15, 2020 at 11:11 AM rohit garg <
> rohit.garg41@gmail.com
> > >
> > > >> > wrote:
> > > >> >
> > > >> > > You can try using kafka manager and check it will fullfill most
> of
> > > >> > > requirement of yours.
> > > >> > >
> > > >> > > Thanks and Regards,
> > > >> > > Rohit
> > > >> > >
> > > >> > > On Wed, Jul 15, 2020, 10:33 Pushkar Deole <pdeole2015@gmail.com
> >
> > > >> wrote:
> > > >> > >
> > > >> > > > Hi All,
> > > >> > > >
> > > >> > > > Any inputs as to how the kafka consumer and producer metrics
> can
> > > be
> > > >> > > hooked
> > > >> > > > up to a monitoring system such as prometheus ?
> > > >> > > >
> > > >> > > > On Tue, Jul 14, 2020 at 4:22 PM Pushkar Deole <
> > > pdeole2015@gmail.com
> > > >> >
> > > >> > > > wrote:
> > > >> > > >
> > > >> > > > > i did find these metrics from confluent docs: however how
> can
> > i
> > > >> > > > > get
> > > >> > > this
> > > >> > > > > metric ? is it available at kafka broker?
> > > >> > > > >
> > > >> > > > > Per-Topic Metrics
> > > >> > > > >
> > > >> > > > > MBean:
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > kafka.producer:type=producer-topic-metrics,client-id=([-.w]+),topic=([
> > > >> > > -.w]+)
> > > >> > > > > record-send-rateThe average number of records sent per
> second
> > > for
> > > >> > > > > a
> > > >> > > > topic.
> > > >> > > > >
> > > >> > > > > On Tue, Jul 14, 2020 at 3:27 PM Pushkar Deole
> > > >> > > > > <pd...@gmail.com>
> > > >> > > > > wrote:
> > > >> > > > >
> > > >> > > > >> Hi All,
> > > >> > > > >>
> > > >> > > > >> Need some help on kafka metrics, i am interested in certain
> > > >> > > > >> metrics
> > > >> > > e.g.
> > > >> > > > >> i need to know the number of records published on a
> > particular
> > > >> > > > >> topic
> > > >> > > and
> > > >> > > > >> number of records consumed from that topic by a specific
> > > consumer
> > > >> > > > group, i
> > > >> > > > >> would need a total of these 2 and also average per second
> for
> > > >> them.
> > > >> > > > >>
> > > >> > > > >> Are those metrics available on kafka brokers and if yes
> then
> > > >> > > > >> which are those metrics that would give me above counts?
> > > >> > > > >>
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > > >
> > >
> >
>
Re: kafka per topic metrics
Posted by Liam Clarke-Hutchinson <li...@adscale.co.nz>.
Hi Pushkar,
Best way to see what metrics are available is to connect to a broker via
JConsole to see the exposed mbeans.
You can iterate over them programmatically by using the MBean API.
Also recommend chapter 10 of Kafka: The Definitive Guide, it covers the
metrics really well.
Cheers,
Liam Clarke-Hutchinson
On Thu, 16 Jul. 2020, 5:05 pm Pushkar Deole, <pd...@gmail.com> wrote:
> Thanks Liam...
> Few questions: in your pattern the topic parameter is appended pattern:
> 'kafka.server<type=BrokerTopicMetrics, name=(.+),
> topic=(.+)><>OneMinuteRate'
> however the kafka docs doesn't mention that
> kafka.server:type=BrokerTopicMetrics,name=MessagesInPerSec
>
> does the topic parameter available in all BrokerTopicMetrics and can the
> broker provide that parameter dynamically by iterating through all topics
> or selective topics? Where will the logic reside to iterate through
> available topics and export metrics for all of them?
>
> On Thu, Jul 16, 2020 at 10:04 AM Liam Clarke-Hutchinson <
> liam.clarke@adscale.co.nz> wrote:
>
> > Whoops, just spotted a typo - the second $1 in the above snippet should
> of
> > course be $2.
> >
> > On Thu, Jul 16, 2020 at 4:33 PM Liam Clarke-Hutchinson <
> > liam.clarke@adscale.co.nz> wrote:
> >
> > > Hi Pushkar,
> > >
> > > There are broker side metrics for messages in / bytes in / bytes out
> per
> > > topic per second. I use this jmx_exporter rule to export them:
> > > - pattern: 'kafka.server<type=BrokerTopicMetrics, name=(.+),
> > > topic=(.+)><>OneMinuteRate'
> > > name: kafka_broker_per_topic_$1_one_minute_rate
> > > labels:
> > > topic: $1
> > > type: GAUGE
> > >
> > > You can't get the number of messages out per topic from the broker
> > > because... I think it's somehow related to batching, or similar, it
> > doesn't
> > > count messages out, only bytes out. You can, however, get that metric
> > from
> > > the consumer if you're using the Java Kafka client, there's a per-topic
> > > messages consumed per second metric exposed as an MBean.
> > >
> > > You could either use jmx_exporter to also export that from the client
> > app,
> > > or if possible, add some code that connects to the mbean inside the JVM
> > and
> > > then exports it via any pre-existing Prometheus registry. You might
> want
> > to
> > > then use a Prometheus aggregating rule to collate all the per-consumer
> > apps
> > > into a per-consumer-group metric, unless the per consumer granularity
> is
> > of
> > > interest to you: https://prometheus.io/docs/practices/rules/
> > >
> > > Hope that helps,
> > >
> > > Kind regards,
> > >
> > > Liam Clarke-Hutchinsons
> > >
> > > On Thu, 16 Jul. 2020, 3:46 pm Pushkar Deole, <pd...@gmail.com>
> > wrote:
> > >
> > >> Thanks Claudia! For broker level metrics, we are also using same jmx
> > >> exporter to export those metrics to prometheus.
> > >> Are you fetching any per topic metrics from broker? e.g. messages
> > produced
> > >> on a certain topic or messages consumed from a certain topic. I am
> > mainly
> > >> interested in these metrics.
> > >>
> > >> I read in kafka docs that they are present at producer/consumer,
> > however I
> > >> am not sure how to fetch them from consumer/producer.
> > >>
> > >> On Wed, Jul 15, 2020 at 8:32 PM Claudia Kesslau <c....@kasasi.de>
> > >> wrote:
> > >>
> > >> > Hi,
> > >> >
> > >> > I use https://github.com/prometheus/jmx_exporter for collecting
> > broker
> > >> > metrics and integrating them into prometheus.
> > >> >
> > >> > Hope this helps.
> > >> > Greetings,
> > >> > Claudia
> > >> >
> > >> > -----Ursprüngliche Nachricht-----
> > >> > Von: Pushkar Deole <pd...@gmail.com>
> > >> > Gesendet: Mittwoch, 15. Juli 2020 09:07
> > >> > An: users@kafka.apache.org
> > >> > Betreff: Re: kafka per topic metrics
> > >> >
> > >> > We are using prometheus as metrics collection and storage system and
> > >> > Grafana for displaying those metrics, so integration with them is
> > >> required
> > >> >
> > >> > On Wed, Jul 15, 2020 at 11:11 AM rohit garg <rohit.garg41@gmail.com
> >
> > >> > wrote:
> > >> >
> > >> > > You can try using kafka manager and check it will fullfill most of
> > >> > > requirement of yours.
> > >> > >
> > >> > > Thanks and Regards,
> > >> > > Rohit
> > >> > >
> > >> > > On Wed, Jul 15, 2020, 10:33 Pushkar Deole <pd...@gmail.com>
> > >> wrote:
> > >> > >
> > >> > > > Hi All,
> > >> > > >
> > >> > > > Any inputs as to how the kafka consumer and producer metrics can
> > be
> > >> > > hooked
> > >> > > > up to a monitoring system such as prometheus ?
> > >> > > >
> > >> > > > On Tue, Jul 14, 2020 at 4:22 PM Pushkar Deole <
> > pdeole2015@gmail.com
> > >> >
> > >> > > > wrote:
> > >> > > >
> > >> > > > > i did find these metrics from confluent docs: however how can
> i
> > >> > > > > get
> > >> > > this
> > >> > > > > metric ? is it available at kafka broker?
> > >> > > > >
> > >> > > > > Per-Topic Metrics
> > >> > > > >
> > >> > > > > MBean:
> > >> > > > >
> > >> > > >
> > >> > >
> > kafka.producer:type=producer-topic-metrics,client-id=([-.w]+),topic=([
> > >> > > -.w]+)
> > >> > > > > record-send-rateThe average number of records sent per second
> > for
> > >> > > > > a
> > >> > > > topic.
> > >> > > > >
> > >> > > > > On Tue, Jul 14, 2020 at 3:27 PM Pushkar Deole
> > >> > > > > <pd...@gmail.com>
> > >> > > > > wrote:
> > >> > > > >
> > >> > > > >> Hi All,
> > >> > > > >>
> > >> > > > >> Need some help on kafka metrics, i am interested in certain
> > >> > > > >> metrics
> > >> > > e.g.
> > >> > > > >> i need to know the number of records published on a
> particular
> > >> > > > >> topic
> > >> > > and
> > >> > > > >> number of records consumed from that topic by a specific
> > consumer
> > >> > > > group, i
> > >> > > > >> would need a total of these 2 and also average per second for
> > >> them.
> > >> > > > >>
> > >> > > > >> Are those metrics available on kafka brokers and if yes then
> > >> > > > >> which are those metrics that would give me above counts?
> > >> > > > >>
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
> >
>
Re: kafka per topic metrics
Posted by Pushkar Deole <pd...@gmail.com>.
Thanks Liam...
Few questions: in your pattern the topic parameter is appended pattern:
'kafka.server<type=BrokerTopicMetrics, name=(.+),
topic=(.+)><>OneMinuteRate'
however the kafka docs doesn't mention that
kafka.server:type=BrokerTopicMetrics,name=MessagesInPerSec
does the topic parameter available in all BrokerTopicMetrics and can the
broker provide that parameter dynamically by iterating through all topics
or selective topics? Where will the logic reside to iterate through
available topics and export metrics for all of them?
On Thu, Jul 16, 2020 at 10:04 AM Liam Clarke-Hutchinson <
liam.clarke@adscale.co.nz> wrote:
> Whoops, just spotted a typo - the second $1 in the above snippet should of
> course be $2.
>
> On Thu, Jul 16, 2020 at 4:33 PM Liam Clarke-Hutchinson <
> liam.clarke@adscale.co.nz> wrote:
>
> > Hi Pushkar,
> >
> > There are broker side metrics for messages in / bytes in / bytes out per
> > topic per second. I use this jmx_exporter rule to export them:
> > - pattern: 'kafka.server<type=BrokerTopicMetrics, name=(.+),
> > topic=(.+)><>OneMinuteRate'
> > name: kafka_broker_per_topic_$1_one_minute_rate
> > labels:
> > topic: $1
> > type: GAUGE
> >
> > You can't get the number of messages out per topic from the broker
> > because... I think it's somehow related to batching, or similar, it
> doesn't
> > count messages out, only bytes out. You can, however, get that metric
> from
> > the consumer if you're using the Java Kafka client, there's a per-topic
> > messages consumed per second metric exposed as an MBean.
> >
> > You could either use jmx_exporter to also export that from the client
> app,
> > or if possible, add some code that connects to the mbean inside the JVM
> and
> > then exports it via any pre-existing Prometheus registry. You might want
> to
> > then use a Prometheus aggregating rule to collate all the per-consumer
> apps
> > into a per-consumer-group metric, unless the per consumer granularity is
> of
> > interest to you: https://prometheus.io/docs/practices/rules/
> >
> > Hope that helps,
> >
> > Kind regards,
> >
> > Liam Clarke-Hutchinsons
> >
> > On Thu, 16 Jul. 2020, 3:46 pm Pushkar Deole, <pd...@gmail.com>
> wrote:
> >
> >> Thanks Claudia! For broker level metrics, we are also using same jmx
> >> exporter to export those metrics to prometheus.
> >> Are you fetching any per topic metrics from broker? e.g. messages
> produced
> >> on a certain topic or messages consumed from a certain topic. I am
> mainly
> >> interested in these metrics.
> >>
> >> I read in kafka docs that they are present at producer/consumer,
> however I
> >> am not sure how to fetch them from consumer/producer.
> >>
> >> On Wed, Jul 15, 2020 at 8:32 PM Claudia Kesslau <c....@kasasi.de>
> >> wrote:
> >>
> >> > Hi,
> >> >
> >> > I use https://github.com/prometheus/jmx_exporter for collecting
> broker
> >> > metrics and integrating them into prometheus.
> >> >
> >> > Hope this helps.
> >> > Greetings,
> >> > Claudia
> >> >
> >> > -----Ursprüngliche Nachricht-----
> >> > Von: Pushkar Deole <pd...@gmail.com>
> >> > Gesendet: Mittwoch, 15. Juli 2020 09:07
> >> > An: users@kafka.apache.org
> >> > Betreff: Re: kafka per topic metrics
> >> >
> >> > We are using prometheus as metrics collection and storage system and
> >> > Grafana for displaying those metrics, so integration with them is
> >> required
> >> >
> >> > On Wed, Jul 15, 2020 at 11:11 AM rohit garg <ro...@gmail.com>
> >> > wrote:
> >> >
> >> > > You can try using kafka manager and check it will fullfill most of
> >> > > requirement of yours.
> >> > >
> >> > > Thanks and Regards,
> >> > > Rohit
> >> > >
> >> > > On Wed, Jul 15, 2020, 10:33 Pushkar Deole <pd...@gmail.com>
> >> wrote:
> >> > >
> >> > > > Hi All,
> >> > > >
> >> > > > Any inputs as to how the kafka consumer and producer metrics can
> be
> >> > > hooked
> >> > > > up to a monitoring system such as prometheus ?
> >> > > >
> >> > > > On Tue, Jul 14, 2020 at 4:22 PM Pushkar Deole <
> pdeole2015@gmail.com
> >> >
> >> > > > wrote:
> >> > > >
> >> > > > > i did find these metrics from confluent docs: however how can i
> >> > > > > get
> >> > > this
> >> > > > > metric ? is it available at kafka broker?
> >> > > > >
> >> > > > > Per-Topic Metrics
> >> > > > >
> >> > > > > MBean:
> >> > > > >
> >> > > >
> >> > >
> kafka.producer:type=producer-topic-metrics,client-id=([-.w]+),topic=([
> >> > > -.w]+)
> >> > > > > record-send-rateThe average number of records sent per second
> for
> >> > > > > a
> >> > > > topic.
> >> > > > >
> >> > > > > On Tue, Jul 14, 2020 at 3:27 PM Pushkar Deole
> >> > > > > <pd...@gmail.com>
> >> > > > > wrote:
> >> > > > >
> >> > > > >> Hi All,
> >> > > > >>
> >> > > > >> Need some help on kafka metrics, i am interested in certain
> >> > > > >> metrics
> >> > > e.g.
> >> > > > >> i need to know the number of records published on a particular
> >> > > > >> topic
> >> > > and
> >> > > > >> number of records consumed from that topic by a specific
> consumer
> >> > > > group, i
> >> > > > >> would need a total of these 2 and also average per second for
> >> them.
> >> > > > >>
> >> > > > >> Are those metrics available on kafka brokers and if yes then
> >> > > > >> which are those metrics that would give me above counts?
> >> > > > >>
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
>
Re: kafka per topic metrics
Posted by Liam Clarke-Hutchinson <li...@adscale.co.nz>.
Whoops, just spotted a typo - the second $1 in the above snippet should of
course be $2.
On Thu, Jul 16, 2020 at 4:33 PM Liam Clarke-Hutchinson <
liam.clarke@adscale.co.nz> wrote:
> Hi Pushkar,
>
> There are broker side metrics for messages in / bytes in / bytes out per
> topic per second. I use this jmx_exporter rule to export them:
> - pattern: 'kafka.server<type=BrokerTopicMetrics, name=(.+),
> topic=(.+)><>OneMinuteRate'
> name: kafka_broker_per_topic_$1_one_minute_rate
> labels:
> topic: $1
> type: GAUGE
>
> You can't get the number of messages out per topic from the broker
> because... I think it's somehow related to batching, or similar, it doesn't
> count messages out, only bytes out. You can, however, get that metric from
> the consumer if you're using the Java Kafka client, there's a per-topic
> messages consumed per second metric exposed as an MBean.
>
> You could either use jmx_exporter to also export that from the client app,
> or if possible, add some code that connects to the mbean inside the JVM and
> then exports it via any pre-existing Prometheus registry. You might want to
> then use a Prometheus aggregating rule to collate all the per-consumer apps
> into a per-consumer-group metric, unless the per consumer granularity is of
> interest to you: https://prometheus.io/docs/practices/rules/
>
> Hope that helps,
>
> Kind regards,
>
> Liam Clarke-Hutchinsons
>
> On Thu, 16 Jul. 2020, 3:46 pm Pushkar Deole, <pd...@gmail.com> wrote:
>
>> Thanks Claudia! For broker level metrics, we are also using same jmx
>> exporter to export those metrics to prometheus.
>> Are you fetching any per topic metrics from broker? e.g. messages produced
>> on a certain topic or messages consumed from a certain topic. I am mainly
>> interested in these metrics.
>>
>> I read in kafka docs that they are present at producer/consumer, however I
>> am not sure how to fetch them from consumer/producer.
>>
>> On Wed, Jul 15, 2020 at 8:32 PM Claudia Kesslau <c....@kasasi.de>
>> wrote:
>>
>> > Hi,
>> >
>> > I use https://github.com/prometheus/jmx_exporter for collecting broker
>> > metrics and integrating them into prometheus.
>> >
>> > Hope this helps.
>> > Greetings,
>> > Claudia
>> >
>> > -----Ursprüngliche Nachricht-----
>> > Von: Pushkar Deole <pd...@gmail.com>
>> > Gesendet: Mittwoch, 15. Juli 2020 09:07
>> > An: users@kafka.apache.org
>> > Betreff: Re: kafka per topic metrics
>> >
>> > We are using prometheus as metrics collection and storage system and
>> > Grafana for displaying those metrics, so integration with them is
>> required
>> >
>> > On Wed, Jul 15, 2020 at 11:11 AM rohit garg <ro...@gmail.com>
>> > wrote:
>> >
>> > > You can try using kafka manager and check it will fullfill most of
>> > > requirement of yours.
>> > >
>> > > Thanks and Regards,
>> > > Rohit
>> > >
>> > > On Wed, Jul 15, 2020, 10:33 Pushkar Deole <pd...@gmail.com>
>> wrote:
>> > >
>> > > > Hi All,
>> > > >
>> > > > Any inputs as to how the kafka consumer and producer metrics can be
>> > > hooked
>> > > > up to a monitoring system such as prometheus ?
>> > > >
>> > > > On Tue, Jul 14, 2020 at 4:22 PM Pushkar Deole <pdeole2015@gmail.com
>> >
>> > > > wrote:
>> > > >
>> > > > > i did find these metrics from confluent docs: however how can i
>> > > > > get
>> > > this
>> > > > > metric ? is it available at kafka broker?
>> > > > >
>> > > > > Per-Topic Metrics
>> > > > >
>> > > > > MBean:
>> > > > >
>> > > >
>> > > kafka.producer:type=producer-topic-metrics,client-id=([-.w]+),topic=([
>> > > -.w]+)
>> > > > > record-send-rateThe average number of records sent per second for
>> > > > > a
>> > > > topic.
>> > > > >
>> > > > > On Tue, Jul 14, 2020 at 3:27 PM Pushkar Deole
>> > > > > <pd...@gmail.com>
>> > > > > wrote:
>> > > > >
>> > > > >> Hi All,
>> > > > >>
>> > > > >> Need some help on kafka metrics, i am interested in certain
>> > > > >> metrics
>> > > e.g.
>> > > > >> i need to know the number of records published on a particular
>> > > > >> topic
>> > > and
>> > > > >> number of records consumed from that topic by a specific consumer
>> > > > group, i
>> > > > >> would need a total of these 2 and also average per second for
>> them.
>> > > > >>
>> > > > >> Are those metrics available on kafka brokers and if yes then
>> > > > >> which are those metrics that would give me above counts?
>> > > > >>
>> > > > >
>> > > >
>> > >
>> >
>>
>
Re: kafka per topic metrics
Posted by Liam Clarke-Hutchinson <li...@adscale.co.nz>.
Hi Pushkar,
There are broker side metrics for messages in / bytes in / bytes out per
topic per second. I use this jmx_exporter rule to export them:
- pattern: 'kafka.server<type=BrokerTopicMetrics, name=(.+),
topic=(.+)><>OneMinuteRate'
name: kafka_broker_per_topic_$1_one_minute_rate
labels:
topic: $1
type: GAUGE
You can't get the number of messages out per topic from the broker
because... I think it's somehow related to batching, or similar, it doesn't
count messages out, only bytes out. You can, however, get that metric from
the consumer if you're using the Java Kafka client, there's a per-topic
messages consumed per second metric exposed as an MBean.
You could either use jmx_exporter to also export that from the client app,
or if possible, add some code that connects to the mbean inside the JVM and
then exports it via any pre-existing Prometheus registry. You might want to
then use a Prometheus aggregating rule to collate all the per-consumer apps
into a per-consumer-group metric, unless the per consumer granularity is of
interest to you: https://prometheus.io/docs/practices/rules/
Hope that helps,
Kind regards,
Liam Clarke-Hutchinsons
On Thu, 16 Jul. 2020, 3:46 pm Pushkar Deole, <pd...@gmail.com> wrote:
> Thanks Claudia! For broker level metrics, we are also using same jmx
> exporter to export those metrics to prometheus.
> Are you fetching any per topic metrics from broker? e.g. messages produced
> on a certain topic or messages consumed from a certain topic. I am mainly
> interested in these metrics.
>
> I read in kafka docs that they are present at producer/consumer, however I
> am not sure how to fetch them from consumer/producer.
>
> On Wed, Jul 15, 2020 at 8:32 PM Claudia Kesslau <c....@kasasi.de>
> wrote:
>
> > Hi,
> >
> > I use https://github.com/prometheus/jmx_exporter for collecting broker
> > metrics and integrating them into prometheus.
> >
> > Hope this helps.
> > Greetings,
> > Claudia
> >
> > -----Ursprüngliche Nachricht-----
> > Von: Pushkar Deole <pd...@gmail.com>
> > Gesendet: Mittwoch, 15. Juli 2020 09:07
> > An: users@kafka.apache.org
> > Betreff: Re: kafka per topic metrics
> >
> > We are using prometheus as metrics collection and storage system and
> > Grafana for displaying those metrics, so integration with them is
> required
> >
> > On Wed, Jul 15, 2020 at 11:11 AM rohit garg <ro...@gmail.com>
> > wrote:
> >
> > > You can try using kafka manager and check it will fullfill most of
> > > requirement of yours.
> > >
> > > Thanks and Regards,
> > > Rohit
> > >
> > > On Wed, Jul 15, 2020, 10:33 Pushkar Deole <pd...@gmail.com>
> wrote:
> > >
> > > > Hi All,
> > > >
> > > > Any inputs as to how the kafka consumer and producer metrics can be
> > > hooked
> > > > up to a monitoring system such as prometheus ?
> > > >
> > > > On Tue, Jul 14, 2020 at 4:22 PM Pushkar Deole <pd...@gmail.com>
> > > > wrote:
> > > >
> > > > > i did find these metrics from confluent docs: however how can i
> > > > > get
> > > this
> > > > > metric ? is it available at kafka broker?
> > > > >
> > > > > Per-Topic Metrics
> > > > >
> > > > > MBean:
> > > > >
> > > >
> > > kafka.producer:type=producer-topic-metrics,client-id=([-.w]+),topic=([
> > > -.w]+)
> > > > > record-send-rateThe average number of records sent per second for
> > > > > a
> > > > topic.
> > > > >
> > > > > On Tue, Jul 14, 2020 at 3:27 PM Pushkar Deole
> > > > > <pd...@gmail.com>
> > > > > wrote:
> > > > >
> > > > >> Hi All,
> > > > >>
> > > > >> Need some help on kafka metrics, i am interested in certain
> > > > >> metrics
> > > e.g.
> > > > >> i need to know the number of records published on a particular
> > > > >> topic
> > > and
> > > > >> number of records consumed from that topic by a specific consumer
> > > > group, i
> > > > >> would need a total of these 2 and also average per second for
> them.
> > > > >>
> > > > >> Are those metrics available on kafka brokers and if yes then
> > > > >> which are those metrics that would give me above counts?
> > > > >>
> > > > >
> > > >
> > >
> >
>
Re: kafka per topic metrics
Posted by Pushkar Deole <pd...@gmail.com>.
Thanks Claudia! For broker level metrics, we are also using same jmx
exporter to export those metrics to prometheus.
Are you fetching any per topic metrics from broker? e.g. messages produced
on a certain topic or messages consumed from a certain topic. I am mainly
interested in these metrics.
I read in kafka docs that they are present at producer/consumer, however I
am not sure how to fetch them from consumer/producer.
On Wed, Jul 15, 2020 at 8:32 PM Claudia Kesslau <c....@kasasi.de> wrote:
> Hi,
>
> I use https://github.com/prometheus/jmx_exporter for collecting broker
> metrics and integrating them into prometheus.
>
> Hope this helps.
> Greetings,
> Claudia
>
> -----Ursprüngliche Nachricht-----
> Von: Pushkar Deole <pd...@gmail.com>
> Gesendet: Mittwoch, 15. Juli 2020 09:07
> An: users@kafka.apache.org
> Betreff: Re: kafka per topic metrics
>
> We are using prometheus as metrics collection and storage system and
> Grafana for displaying those metrics, so integration with them is required
>
> On Wed, Jul 15, 2020 at 11:11 AM rohit garg <ro...@gmail.com>
> wrote:
>
> > You can try using kafka manager and check it will fullfill most of
> > requirement of yours.
> >
> > Thanks and Regards,
> > Rohit
> >
> > On Wed, Jul 15, 2020, 10:33 Pushkar Deole <pd...@gmail.com> wrote:
> >
> > > Hi All,
> > >
> > > Any inputs as to how the kafka consumer and producer metrics can be
> > hooked
> > > up to a monitoring system such as prometheus ?
> > >
> > > On Tue, Jul 14, 2020 at 4:22 PM Pushkar Deole <pd...@gmail.com>
> > > wrote:
> > >
> > > > i did find these metrics from confluent docs: however how can i
> > > > get
> > this
> > > > metric ? is it available at kafka broker?
> > > >
> > > > Per-Topic Metrics
> > > >
> > > > MBean:
> > > >
> > >
> > kafka.producer:type=producer-topic-metrics,client-id=([-.w]+),topic=([
> > -.w]+)
> > > > record-send-rateThe average number of records sent per second for
> > > > a
> > > topic.
> > > >
> > > > On Tue, Jul 14, 2020 at 3:27 PM Pushkar Deole
> > > > <pd...@gmail.com>
> > > > wrote:
> > > >
> > > >> Hi All,
> > > >>
> > > >> Need some help on kafka metrics, i am interested in certain
> > > >> metrics
> > e.g.
> > > >> i need to know the number of records published on a particular
> > > >> topic
> > and
> > > >> number of records consumed from that topic by a specific consumer
> > > group, i
> > > >> would need a total of these 2 and also average per second for them.
> > > >>
> > > >> Are those metrics available on kafka brokers and if yes then
> > > >> which are those metrics that would give me above counts?
> > > >>
> > > >
> > >
> >
>
AW: kafka per topic metrics
Posted by Claudia Kesslau <c....@kasasi.de>.
Hi,
I use https://github.com/prometheus/jmx_exporter for collecting broker metrics and integrating them into prometheus.
Hope this helps.
Greetings,
Claudia
-----Ursprüngliche Nachricht-----
Von: Pushkar Deole <pd...@gmail.com>
Gesendet: Mittwoch, 15. Juli 2020 09:07
An: users@kafka.apache.org
Betreff: Re: kafka per topic metrics
We are using prometheus as metrics collection and storage system and Grafana for displaying those metrics, so integration with them is required
On Wed, Jul 15, 2020 at 11:11 AM rohit garg <ro...@gmail.com> wrote:
> You can try using kafka manager and check it will fullfill most of
> requirement of yours.
>
> Thanks and Regards,
> Rohit
>
> On Wed, Jul 15, 2020, 10:33 Pushkar Deole <pd...@gmail.com> wrote:
>
> > Hi All,
> >
> > Any inputs as to how the kafka consumer and producer metrics can be
> hooked
> > up to a monitoring system such as prometheus ?
> >
> > On Tue, Jul 14, 2020 at 4:22 PM Pushkar Deole <pd...@gmail.com>
> > wrote:
> >
> > > i did find these metrics from confluent docs: however how can i
> > > get
> this
> > > metric ? is it available at kafka broker?
> > >
> > > Per-Topic Metrics
> > >
> > > MBean:
> > >
> >
> kafka.producer:type=producer-topic-metrics,client-id=([-.w]+),topic=([
> -.w]+)
> > > record-send-rateThe average number of records sent per second for
> > > a
> > topic.
> > >
> > > On Tue, Jul 14, 2020 at 3:27 PM Pushkar Deole
> > > <pd...@gmail.com>
> > > wrote:
> > >
> > >> Hi All,
> > >>
> > >> Need some help on kafka metrics, i am interested in certain
> > >> metrics
> e.g.
> > >> i need to know the number of records published on a particular
> > >> topic
> and
> > >> number of records consumed from that topic by a specific consumer
> > group, i
> > >> would need a total of these 2 and also average per second for them.
> > >>
> > >> Are those metrics available on kafka brokers and if yes then
> > >> which are those metrics that would give me above counts?
> > >>
> > >
> >
>
Re: kafka per topic metrics
Posted by Pushkar Deole <pd...@gmail.com>.
We are using prometheus as metrics collection and storage system and
Grafana for displaying those metrics, so integration with them is required
On Wed, Jul 15, 2020 at 11:11 AM rohit garg <ro...@gmail.com> wrote:
> You can try using kafka manager and check it will fullfill most of
> requirement of yours.
>
> Thanks and Regards,
> Rohit
>
> On Wed, Jul 15, 2020, 10:33 Pushkar Deole <pd...@gmail.com> wrote:
>
> > Hi All,
> >
> > Any inputs as to how the kafka consumer and producer metrics can be
> hooked
> > up to a monitoring system such as prometheus ?
> >
> > On Tue, Jul 14, 2020 at 4:22 PM Pushkar Deole <pd...@gmail.com>
> > wrote:
> >
> > > i did find these metrics from confluent docs: however how can i get
> this
> > > metric ? is it available at kafka broker?
> > >
> > > Per-Topic Metrics
> > >
> > > MBean:
> > >
> >
> kafka.producer:type=producer-topic-metrics,client-id=([-.w]+),topic=([-.w]+)
> > > record-send-rateThe average number of records sent per second for a
> > topic.
> > >
> > > On Tue, Jul 14, 2020 at 3:27 PM Pushkar Deole <pd...@gmail.com>
> > > wrote:
> > >
> > >> Hi All,
> > >>
> > >> Need some help on kafka metrics, i am interested in certain metrics
> e.g.
> > >> i need to know the number of records published on a particular topic
> and
> > >> number of records consumed from that topic by a specific consumer
> > group, i
> > >> would need a total of these 2 and also average per second for them.
> > >>
> > >> Are those metrics available on kafka brokers and if yes then which are
> > >> those metrics that would give me above counts?
> > >>
> > >
> >
>
Re: kafka per topic metrics
Posted by rohit garg <ro...@gmail.com>.
You can try using kafka manager and check it will fullfill most of
requirement of yours.
Thanks and Regards,
Rohit
On Wed, Jul 15, 2020, 10:33 Pushkar Deole <pd...@gmail.com> wrote:
> Hi All,
>
> Any inputs as to how the kafka consumer and producer metrics can be hooked
> up to a monitoring system such as prometheus ?
>
> On Tue, Jul 14, 2020 at 4:22 PM Pushkar Deole <pd...@gmail.com>
> wrote:
>
> > i did find these metrics from confluent docs: however how can i get this
> > metric ? is it available at kafka broker?
> >
> > Per-Topic Metrics
> >
> > MBean:
> >
> kafka.producer:type=producer-topic-metrics,client-id=([-.w]+),topic=([-.w]+)
> > record-send-rateThe average number of records sent per second for a
> topic.
> >
> > On Tue, Jul 14, 2020 at 3:27 PM Pushkar Deole <pd...@gmail.com>
> > wrote:
> >
> >> Hi All,
> >>
> >> Need some help on kafka metrics, i am interested in certain metrics e.g.
> >> i need to know the number of records published on a particular topic and
> >> number of records consumed from that topic by a specific consumer
> group, i
> >> would need a total of these 2 and also average per second for them.
> >>
> >> Are those metrics available on kafka brokers and if yes then which are
> >> those metrics that would give me above counts?
> >>
> >
>
Re: kafka per topic metrics
Posted by Pushkar Deole <pd...@gmail.com>.
Hi All,
Any inputs as to how the kafka consumer and producer metrics can be hooked
up to a monitoring system such as prometheus ?
On Tue, Jul 14, 2020 at 4:22 PM Pushkar Deole <pd...@gmail.com> wrote:
> i did find these metrics from confluent docs: however how can i get this
> metric ? is it available at kafka broker?
>
> Per-Topic Metrics
>
> MBean:
> kafka.producer:type=producer-topic-metrics,client-id=([-.w]+),topic=([-.w]+)
> record-send-rateThe average number of records sent per second for a topic.
>
> On Tue, Jul 14, 2020 at 3:27 PM Pushkar Deole <pd...@gmail.com>
> wrote:
>
>> Hi All,
>>
>> Need some help on kafka metrics, i am interested in certain metrics e.g.
>> i need to know the number of records published on a particular topic and
>> number of records consumed from that topic by a specific consumer group, i
>> would need a total of these 2 and also average per second for them.
>>
>> Are those metrics available on kafka brokers and if yes then which are
>> those metrics that would give me above counts?
>>
>
Re: kafka per topic metrics
Posted by Pushkar Deole <pd...@gmail.com>.
i did find these metrics from confluent docs: however how can i get this
metric ? is it available at kafka broker?
Per-Topic Metrics
MBean:
kafka.producer:type=producer-topic-metrics,client-id=([-.w]+),topic=([-.w]+)
record-send-rateThe average number of records sent per second for a topic.
On Tue, Jul 14, 2020 at 3:27 PM Pushkar Deole <pd...@gmail.com> wrote:
> Hi All,
>
> Need some help on kafka metrics, i am interested in certain metrics e.g. i
> need to know the number of records published on a particular topic and
> number of records consumed from that topic by a specific consumer group, i
> would need a total of these 2 and also average per second for them.
>
> Are those metrics available on kafka brokers and if yes then which are
> those metrics that would give me above counts?
>