How to emit lag into Prometheus?

[Dumitru Nicolae Marasoiu <ni...@gmail.com>]

Hi, what is suggested way to emit lag into Prometheus? Is there a Kafka
Streams job for it? Is Burrow or Kafka Lag Exporter?
Emitting lag from the consumer is not a great option of course, so looking
at independent jobs that emit lag.
Do you know any? Thank you

Re: How to emit lag into Prometheus?

[Ran Lupovich <ra...@gmail.com>]

Did you test this out
https://github.com/lightbend/kafka-lag-exporter
?

We are using bash scripting to describe the consumer groups from the
cluster side into file and ingest to splunk for dashboarsing...

Note - Approach of describing from outside the consumer does not give the
'real time' view but only the 'comitted' view

בתאריך יום ו׳, 23 באפר׳ 2021, 12:20, מאת Dumitru Nicolae Marasoiu ‏<
nicolae.marasoiu@gmail.com>:

> Hi, what is suggested way to emit lag into Prometheus? Is there a Kafka
> Streams job for it? Is Burrow or Kafka Lag Exporter?
> Emitting lag from the consumer is not a great option of course, so looking
> at independent jobs that emit lag.
> Do you know any? Thank you
>

Re: How to emit lag into Prometheus?

[Jonas Seydel <jo...@s-seydel.de>]

Hi,

we're using Kafka Minion (named KMinion since v2) for this purpose: 
https://github.com/cloudhut/kminion
It can export all kinds of metrics for a cluster and consumer groups, 
including their lag.

Regards,
Jonas Seydel


On 23.04.21 11:20, Dumitru Nicolae Marasoiu wrote:
> Hi, what is suggested way to emit lag into Prometheus? Is there a Kafka
> Streams job for it? Is Burrow or Kafka Lag Exporter?
> Emitting lag from the consumer is not a great option of course, so looking
> at independent jobs that emit lag.
> Do you know any? Thank you
>