Kafka topology monitoring

[ti...@milliken-consultancy.com]

Good Afternoon and hope you all are well. 

Im trying to monitor my topologies to ensure That I can look at the
following metrics 

Thread metrics 

- Average time for commits, poll, process operations 

- Tasks created per second, tasked closed per second 

I have seen metrics in micrometer, but they don't seem to extrapolate
the above messages, is there a tool that I can use to get this level of
detailed monitoring? 

best regards 

Timothy

Re: Kafka topology monitoring

[Guozhang Wang <wa...@gmail.com>]

Hello Timothy,

I'm not sure I get your question: what do you mean that these metrics `do
not seem to extrapolate the above messages`?

Note that the metrics reporting has reporting levels (current there are
INFO and DEBUG) and the ones you mentioned are DEBUG level, so you'd need
to override your metrics reporting level config to DEBUG, details here:

https://cwiki.apache.org/confluence/display/KAFKA/KIP-444%3A+Augment+metrics+for+Kafka+Streams

Guozhang


On Mon, Apr 13, 2020 at 8:14 AM <ti...@milliken-consultancy.com> wrote:

> Good Afternoon and hope you all are well.
>
> Im trying to monitor my topologies to ensure That I can look at the
> following metrics
>
> Thread metrics
>
> - Average time for commits, poll, process operations
>
> - Tasks created per second, tasked closed per second
>
> I have seen metrics in micrometer, but they don't seem to extrapolate
> the above messages, is there a tool that I can use to get this level of
> detailed monitoring?
>
> best regards
>
> Timothy



-- 
-- Guozhang