You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Anandh Kumar <an...@gmail.com> on 2015/07/10 05:53:46 UTC
How to monitor Kafka offset
Hi
We have any monitor tool which monitor kafka offset?
I need some opensource admin tool for kafka.
Please guide me.
Regards,
-Anandh Kumar
Re: How to monitor Kafka offset
Posted by Adam Dubiel <du...@gmail.com>.
As i mentioned earlier, this feature has not yet been released, but out
pull-request has been approved by Quantifind. Here it is:
https://github.com/quantifind/KafkaOffsetMonitor/pull/58/files
If you would like to use it now, you would have to build Offset Monitor jar
yourself using code from fork. Otherwise you will need to wait for
Quantifind to merge & release new version.
This is repository of Graphite integration plugin:
https://github.com/allegro/kafka-offset-monitor-graphite
We are already using it on production, works great and gives us tons of
data to monitor.
Having data in Graphite, you can use some Graphite monitoring tool like
Cabot (https://github.com/arachnys/cabot), that triggers alerts based on
stuff from Graphite.
Adam
2015-07-10 10:45 GMT+02:00 Anandh Kumar <an...@gmail.com>:
> Adam,
>
> Tried to config KafkaOffsetMonitor it is working fine.
>
> But how can we integrated with Graphite to add alerting ? can please
> explain in details and if you have any doc can you please provide?
>
> Thanks & Regards,
> -Anandh Kumar
>
> On Fri, Jul 10, 2015 at 12:58 PM, Adam Dubiel <du...@gmail.com>
> wrote:
>
> > We are using kafka offset monitor (
> > http://quantifind.com/KafkaOffsetMonitor/),
> > which we recently integrated with Graphite to add alerting and better
> > graphing - it should be accessible in newest version, not yet released.
> It
> > works only with ZK offsets though.
> >
> > 2015-07-10 9:24 GMT+02:00 Anandh Kumar <an...@gmail.com>:
> >
> > > Thanks Rahul for your reply
> > >
> > > On Fri, Jul 10, 2015 at 11:11 AM, Rahul Jain <ra...@gmail.com>
> wrote:
> > >
> > > > Burrow works only if you are storing the offsets in kafka topic, not
> > > > zookeeper. You can also take a look at Kafka web console ( it has a
> > > memory
> > > > leak bug but a patch is available ).
> > > > On 10 Jul 2015 09:34, "Jiangjie Qin" <jq...@linkedin.com.invalid>
> > wrote:
> > > >
> > > > > You can take a look at Burrow. We use it in LinkedIn.
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Jiangjie (Becket) Qin
> > > > >
> > > > > On 7/9/15, 8:53 PM, "Anandh Kumar" <an...@gmail.com> wrote:
> > > > >
> > > > > >Hi
> > > > > >
> > > > > >We have any monitor tool which monitor kafka offset?
> > > > > >
> > > > > >I need some opensource admin tool for kafka.
> > > > > >
> > > > > >Please guide me.
> > > > > >
> > > > > >Regards,
> > > > > >-Anandh Kumar
> > > > >
> > > > >
> > > >
> > >
> >
>
Re: How to monitor Kafka offset
Posted by Anandh Kumar <an...@gmail.com>.
Adam,
Tried to config KafkaOffsetMonitor it is working fine.
But how can we integrated with Graphite to add alerting ? can please
explain in details and if you have any doc can you please provide?
Thanks & Regards,
-Anandh Kumar
On Fri, Jul 10, 2015 at 12:58 PM, Adam Dubiel <du...@gmail.com> wrote:
> We are using kafka offset monitor (
> http://quantifind.com/KafkaOffsetMonitor/),
> which we recently integrated with Graphite to add alerting and better
> graphing - it should be accessible in newest version, not yet released. It
> works only with ZK offsets though.
>
> 2015-07-10 9:24 GMT+02:00 Anandh Kumar <an...@gmail.com>:
>
> > Thanks Rahul for your reply
> >
> > On Fri, Jul 10, 2015 at 11:11 AM, Rahul Jain <ra...@gmail.com> wrote:
> >
> > > Burrow works only if you are storing the offsets in kafka topic, not
> > > zookeeper. You can also take a look at Kafka web console ( it has a
> > memory
> > > leak bug but a patch is available ).
> > > On 10 Jul 2015 09:34, "Jiangjie Qin" <jq...@linkedin.com.invalid>
> wrote:
> > >
> > > > You can take a look at Burrow. We use it in LinkedIn.
> > > >
> > > > Thanks,
> > > >
> > > > Jiangjie (Becket) Qin
> > > >
> > > > On 7/9/15, 8:53 PM, "Anandh Kumar" <an...@gmail.com> wrote:
> > > >
> > > > >Hi
> > > > >
> > > > >We have any monitor tool which monitor kafka offset?
> > > > >
> > > > >I need some opensource admin tool for kafka.
> > > > >
> > > > >Please guide me.
> > > > >
> > > > >Regards,
> > > > >-Anandh Kumar
> > > >
> > > >
> > >
> >
>
Re: How to monitor Kafka offset
Posted by Adam Dubiel <du...@gmail.com>.
We are using kafka offset monitor (http://quantifind.com/KafkaOffsetMonitor/),
which we recently integrated with Graphite to add alerting and better
graphing - it should be accessible in newest version, not yet released. It
works only with ZK offsets though.
2015-07-10 9:24 GMT+02:00 Anandh Kumar <an...@gmail.com>:
> Thanks Rahul for your reply
>
> On Fri, Jul 10, 2015 at 11:11 AM, Rahul Jain <ra...@gmail.com> wrote:
>
> > Burrow works only if you are storing the offsets in kafka topic, not
> > zookeeper. You can also take a look at Kafka web console ( it has a
> memory
> > leak bug but a patch is available ).
> > On 10 Jul 2015 09:34, "Jiangjie Qin" <jq...@linkedin.com.invalid> wrote:
> >
> > > You can take a look at Burrow. We use it in LinkedIn.
> > >
> > > Thanks,
> > >
> > > Jiangjie (Becket) Qin
> > >
> > > On 7/9/15, 8:53 PM, "Anandh Kumar" <an...@gmail.com> wrote:
> > >
> > > >Hi
> > > >
> > > >We have any monitor tool which monitor kafka offset?
> > > >
> > > >I need some opensource admin tool for kafka.
> > > >
> > > >Please guide me.
> > > >
> > > >Regards,
> > > >-Anandh Kumar
> > >
> > >
> >
>
Re: How to monitor Kafka offset
Posted by Anandh Kumar <an...@gmail.com>.
Thanks Rahul for your reply
On Fri, Jul 10, 2015 at 11:11 AM, Rahul Jain <ra...@gmail.com> wrote:
> Burrow works only if you are storing the offsets in kafka topic, not
> zookeeper. You can also take a look at Kafka web console ( it has a memory
> leak bug but a patch is available ).
> On 10 Jul 2015 09:34, "Jiangjie Qin" <jq...@linkedin.com.invalid> wrote:
>
> > You can take a look at Burrow. We use it in LinkedIn.
> >
> > Thanks,
> >
> > Jiangjie (Becket) Qin
> >
> > On 7/9/15, 8:53 PM, "Anandh Kumar" <an...@gmail.com> wrote:
> >
> > >Hi
> > >
> > >We have any monitor tool which monitor kafka offset?
> > >
> > >I need some opensource admin tool for kafka.
> > >
> > >Please guide me.
> > >
> > >Regards,
> > >-Anandh Kumar
> >
> >
>
Re: How to monitor Kafka offset
Posted by Rahul Jain <ra...@gmail.com>.
Burrow works only if you are storing the offsets in kafka topic, not
zookeeper. You can also take a look at Kafka web console ( it has a memory
leak bug but a patch is available ).
On 10 Jul 2015 09:34, "Jiangjie Qin" <jq...@linkedin.com.invalid> wrote:
> You can take a look at Burrow. We use it in LinkedIn.
>
> Thanks,
>
> Jiangjie (Becket) Qin
>
> On 7/9/15, 8:53 PM, "Anandh Kumar" <an...@gmail.com> wrote:
>
> >Hi
> >
> >We have any monitor tool which monitor kafka offset?
> >
> >I need some opensource admin tool for kafka.
> >
> >Please guide me.
> >
> >Regards,
> >-Anandh Kumar
>
>
Re: How to monitor Kafka offset
Posted by Jiangjie Qin <jq...@linkedin.com.INVALID>.
You can take a look at Burrow. We use it in LinkedIn.
Thanks,
Jiangjie (Becket) Qin
On 7/9/15, 8:53 PM, "Anandh Kumar" <an...@gmail.com> wrote:
>Hi
>
>We have any monitor tool which monitor kafka offset?
>
>I need some opensource admin tool for kafka.
>
>Please guide me.
>
>Regards,
>-Anandh Kumar