You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by "JOHN, BIBIN" <bj...@att.com> on 2020/01/10 04:05:19 UTC
JMX Metrics to display disk Usage
Is there any metrics available to view disk usage in each broker/topic via JMX exporter? If yes, could you please send me details? Thanks
RE: JMX Metrics to display disk Usage
Posted by "JOHN, BIBIN" <bj...@att.com>.
I already have Prometheus/Grafana for monitoring for most of the metrics. But I was looking for metrics which shows disk usage per broker/topic.
-----Original Message-----
From: George <ge...@gmail.com>
Sent: Thursday, January 9, 2020 10:49 PM
To: users@kafka.apache.org
Subject: Re: JMX Metrics to display disk Usage
Why not rather use Prometheus with node-app module deployed on all involved hosts, Producers, consumers, brokers etc,
You can also use prometheus to integrate with Kafka via JMX to expose it's kafka metrics,
Display all via a Grafana dashboard
G
On Fri, Jan 10, 2020 at 6:21 AM Peter Bukowinski <pm...@gmail.com> wrote:
> Kafka does not report collect/report on topic data filesystem usage. I
> used this collectd project to help me collect the topic usage data and
> export it to graphite:
>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_HubSpo
> t_collectd-2Dkafka-2Ddisk_blob_master_README.md&d=DwIFaQ&c=LFYZ-o9_HUM
> eMTSQicvjIg&r=eFshZuDXOwvmW_UjVcAH8Q&m=3sUZLopS5LwrfOAIKf6w2dm6Ssbx9Nd
> 5IpnU8C4JGw0&s=KG_PegcYUL_-2T6gatmtIJevJYypkc_SYHVYpRSPIUU&e=
>
> The plugin collects the size of each topic-partition directory on disk.
> From graphite, I sum that data per topic so I can see total disk usage
> per topic across my cluster.
>
> -- Peter
>
> > On Jan 9, 2020, at 8:05 PM, JOHN, BIBIN <bj...@att.com> wrote:
> >
> > Is there any metrics available to view disk usage in each
> > broker/topic
> via JMX exporter? If yes, could you please send me details? Thanks
>
--
You have the obligation to inform one honestly of the risk, and as a person you are committed to educate yourself to the total risk in any activity!
Once informed & totally aware of the risk, every fool has the right to kill or injure themselves as they see fit!
Re: JMX Metrics to display disk Usage
Posted by George <ge...@gmail.com>.
Why not rather use Prometheus with node-app module deployed on all involved
hosts, Producers, consumers, brokers etc,
You can also use prometheus to integrate with Kafka via JMX to expose it's
kafka metrics,
Display all via a Grafana dashboard
G
On Fri, Jan 10, 2020 at 6:21 AM Peter Bukowinski <pm...@gmail.com> wrote:
> Kafka does not report collect/report on topic data filesystem usage. I
> used this collectd project to help me collect the topic usage data and
> export it to graphite:
>
> https://github.com/HubSpot/collectd-kafka-disk/blob/master/README.md
>
> The plugin collects the size of each topic-partition directory on disk.
> From graphite, I sum that data per topic so I can see total disk usage per
> topic across my cluster.
>
> -- Peter
>
> > On Jan 9, 2020, at 8:05 PM, JOHN, BIBIN <bj...@att.com> wrote:
> >
> > Is there any metrics available to view disk usage in each broker/topic
> via JMX exporter? If yes, could you please send me details? Thanks
>
--
You have the obligation to inform one honestly of the risk, and as a person
you are committed to educate yourself to the total risk in any activity!
Once informed & totally aware of the risk,
every fool has the right to kill or injure themselves as they see fit!
Re: JMX Metrics to display disk Usage
Posted by Peter Bukowinski <pm...@gmail.com>.
Kafka does not report collect/report on topic data filesystem usage. I used this collectd project to help me collect the topic usage data and export it to graphite:
https://github.com/HubSpot/collectd-kafka-disk/blob/master/README.md
The plugin collects the size of each topic-partition directory on disk. From graphite, I sum that data per topic so I can see total disk usage per topic across my cluster.
-- Peter
> On Jan 9, 2020, at 8:05 PM, JOHN, BIBIN <bj...@att.com> wrote:
>
> Is there any metrics available to view disk usage in each broker/topic via JMX exporter? If yes, could you please send me details? Thanks