You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Rajiv Jivan <ra...@yahoo.com.INVALID> on 2015/12/18 21:37:11 UTC
Monitoring MirrorMaker
I have a couple of questions on how to monitor MirrorMaker using ConsumerOffsetChecker.
1. When viewing a topic I see multiple rows. Is each row for one partition
2. I am looking to write a nagios plugin to alert if MirrorMaker is running but isn't keeping up. For example there is a network connectivity issue. Is summing up the lag and triggering alerts based on the value recommended.
Also curios to know what other folks have been doing.
Regards,
Rajiv
Re: Monitoring MirrorMaker
Posted by Rajiv Jivan <ra...@yahoo.com.INVALID>.
Thanks for the pointers Pablo. Burrow looks like a really good fit. I'll update this list with my findings.
On Friday, December 18, 2015 8:54 PM, Pablo Fischer <pa...@pablo.com.mx> wrote:
We ran through this a few months ago, here is a list of things and tools
I'd recommend you:
- install burrow. It monitors the consumers and make sure they are not
lagging behind, it also covers other corner cases that can get tricky with
the offset checker. We query burrow (has an API) and then generate alerts
tho it also has email/alert support.
- not sure how it is on 0.9 (we are still on 0.8.2) but we use a different
mirrormaker that preserves partitions. If I remember correctly when we used
the official one we found it was mirroring everything to a single partition
:/. But regardless, we also query the logs and look for errors and report
errors and even auto restart the process if needed.
On Friday, December 18, 2015, Rajiv Jivan <ra...@yahoo.com.invalid> wrote:
> I have a couple of questions on how to monitor MirrorMaker using
> ConsumerOffsetChecker.
> 1. When viewing a topic I see multiple rows. Is each row for one partition
> 2. I am looking to write a nagios plugin to alert if MirrorMaker is
> running but isn't keeping up. For example there is a network connectivity
> issue. Is summing up the lag and triggering alerts based on the value
> recommended.
>
> Also curios to know what other folks have been doing.
>
> Regards,
>
> Rajiv
>
--
Pablo
Re: Monitoring MirrorMaker
Posted by Pablo Fischer <pa...@pablo.com.mx>.
We ran through this a few months ago, here is a list of things and tools
I'd recommend you:
- install burrow. It monitors the consumers and make sure they are not
lagging behind, it also covers other corner cases that can get tricky with
the offset checker. We query burrow (has an API) and then generate alerts
tho it also has email/alert support.
- not sure how it is on 0.9 (we are still on 0.8.2) but we use a different
mirrormaker that preserves partitions. If I remember correctly when we used
the official one we found it was mirroring everything to a single partition
:/. But regardless, we also query the logs and look for errors and report
errors and even auto restart the process if needed.
On Friday, December 18, 2015, Rajiv Jivan <ra...@yahoo.com.invalid> wrote:
> I have a couple of questions on how to monitor MirrorMaker using
> ConsumerOffsetChecker.
> 1. When viewing a topic I see multiple rows. Is each row for one partition
> 2. I am looking to write a nagios plugin to alert if MirrorMaker is
> running but isn't keeping up. For example there is a network connectivity
> issue. Is summing up the lag and triggering alerts based on the value
> recommended.
>
> Also curios to know what other folks have been doing.
>
> Regards,
>
> Rajiv
>
--
Pablo