You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Otis Gospodnetic <ot...@gmail.com> on 2013/05/07 22:45:55 UTC
Kafka Monitoring, 0.7 vs. 0.8 JMX
Hi,
We're considering adding Kafka monitoring to SPM (see
http://sematext.com/spm/index.html ). We use Kafka ourselves and
would like to see our Kafka metrics in SPM along with Hadoop, HBase,
Solr, and other metrics we monitor.
My questions:
* What do people currently use for monitoring Kafka?
* Is there apetite for adding Kafka support to SPM?
* How different is the structure/content of JMX for 0.7 vs. 0.8? Is
0.8 a backwards-compatible superset of 0.7?
Thanks,
Otis
--
Search Analytics - http://sematext.com/search-analytics/index.html
Performance Monitoring - http://sematext.com/spm/index.html
Re: Kafka Monitoring, 0.7 vs. 0.8 JMX
Posted by Jun Rao <ju...@gmail.com>.
See http://metrics.codahale.com/getting-started/#reporting-via-http
Thanks,
Jun
On Wed, May 8, 2013 at 9:03 AM, Dennis Haller <dh...@talenttech.com>wrote:
> What exactly is a metric reporter - something in log4j?
>
> Thanks
> Dennis
>
>
>
> On Wed, May 8, 2013 at 8:20 AM, Jun Rao <ju...@gmail.com> wrote:
>
> > 0.8 JMX is different from 0.7. In 0.8, all jmx beans are exposed through
> > metrics. One can attach a metric reporter for monitoring.
> >
> > Thanks,
> >
> > Jun
> >
> >
> > On Tue, May 7, 2013 at 1:45 PM, Otis Gospodnetic <
> > otis.gospodnetic@gmail.com
> > > wrote:
> >
> > > Hi,
> > >
> > > We're considering adding Kafka monitoring to SPM (see
> > > http://sematext.com/spm/index.html ). We use Kafka ourselves and
> > > would like to see our Kafka metrics in SPM along with Hadoop, HBase,
> > > Solr, and other metrics we monitor.
> > >
> > > My questions:
> > > * What do people currently use for monitoring Kafka?
> > > * Is there apetite for adding Kafka support to SPM?
> > > * How different is the structure/content of JMX for 0.7 vs. 0.8? Is
> > > 0.8 a backwards-compatible superset of 0.7?
> > >
> > > Thanks,
> > > Otis
> > > --
> > > Search Analytics - http://sematext.com/search-analytics/index.html
> > > Performance Monitoring - http://sematext.com/spm/index.html
> > >
> >
>
Re: Kafka Monitoring, 0.7 vs. 0.8 JMX
Posted by Dennis Haller <dh...@talenttech.com>.
What exactly is a metric reporter - something in log4j?
Thanks
Dennis
On Wed, May 8, 2013 at 8:20 AM, Jun Rao <ju...@gmail.com> wrote:
> 0.8 JMX is different from 0.7. In 0.8, all jmx beans are exposed through
> metrics. One can attach a metric reporter for monitoring.
>
> Thanks,
>
> Jun
>
>
> On Tue, May 7, 2013 at 1:45 PM, Otis Gospodnetic <
> otis.gospodnetic@gmail.com
> > wrote:
>
> > Hi,
> >
> > We're considering adding Kafka monitoring to SPM (see
> > http://sematext.com/spm/index.html ). We use Kafka ourselves and
> > would like to see our Kafka metrics in SPM along with Hadoop, HBase,
> > Solr, and other metrics we monitor.
> >
> > My questions:
> > * What do people currently use for monitoring Kafka?
> > * Is there apetite for adding Kafka support to SPM?
> > * How different is the structure/content of JMX for 0.7 vs. 0.8? Is
> > 0.8 a backwards-compatible superset of 0.7?
> >
> > Thanks,
> > Otis
> > --
> > Search Analytics - http://sematext.com/search-analytics/index.html
> > Performance Monitoring - http://sematext.com/spm/index.html
> >
>
Re: Kafka Monitoring, 0.7 vs. 0.8 JMX
Posted by David DeMaagd <dd...@linkedin.com>.
I think there's really two angles to look at this from...
1) What is 'important' to monitor? Meaning, what subset of these are
important/critical for being able to tell system health (things you want
to set alerts on), what subset are nice to have for overall health and
capacity planning (things you want to create pretty graphs on) and the
rest (not immediately useful in general, but can really help in a
debugging/triage situation).
2) How do you get the data? Kind of independent of the above, though
kinda related as well.
As for the second one, you need to look at the collection mechanics. As
you mentioned below, large scale polling (especially with a non-trivial
number of beans) is expensive and problematic no matter how you do it
(JMX or HTTP) given enough scale. I don't have much experience with the
codahale metrics route directly, but I have messed with Jolokia, which is
likely in the same boat - they expose the metrics for you to grab. In
both cases, given enough data points (and kafka, depending on the number
of topics involved, has a /lot/ of them), either can be slow if not
implemented carefully. Meaning you may overrun your desired polling
interval.
In very large environments, I've found it very scalable to have either a
local poller on the box (which could be reading via JMX or HTTP) which
then emits the data to something or have some kind of wrapper around the
application that does the collection/emission (launching the broker as a
thread, and the parent process dows some JMX magic to connect to the
data points). Both of these routes depend a lot on your monitoring
infrastructure, but they will help you get around the general wide
polling problem...
Semi-shameless plug for how it is done at LinkedIn - http://engineering.linkedin.com/52/autometrics-self-service-metrics-collection
--
Dave DeMaagd
ddemaagd@linkedin.com | 818 262 7958
(Dragos.Manolescu@servicenow.com - Wed, May 08, 2013 at 09:27:21PM +0000)
> From the JmxReporter section of the metrics manual:
>
> Warning
> We don¹t recommend that you try to gather metrics from your production
> environment. JMX¹s RPC API is fragile and bonkers. For development
> purposes and browsing, though, it can be very useful.
>
>
>
> -Dragos
>
> On 5/8/13 2:10 PM, "Otis Gospodnetic" <ot...@yahoo.com> wrote:
>
> >Also, do you recommend getting metrics via JMX or via HTTP?
>
Re: Kafka Monitoring, 0.7 vs. 0.8 JMX
Posted by Jun Rao <ju...@gmail.com>.
There is also a Ganglia reporter for metrics. Don't know how much overhead
it adds though.
Thanks,
Jun
On Wed, May 8, 2013 at 2:10 PM, Otis Gospodnetic <otis_gospodnetic@yahoo.com
> wrote:
> Hi Jun,
>
> Does that imply that what 0.8 puts in JMX is a superset of what's in JMX
> in 0.7?
> Or have names or types of beans changed?
>
> Also, do you recommend getting metrics via JMX or via HTTP?
>
> Thanks,
> Otis
> ----
> Monitoring for Solr / ElasticSearch / HBase / Hadoop -
> http://sematext.com/spm
>
>
>
>
> >________________________________
> > From: Jun Rao <ju...@gmail.com>
> >To: "users@kafka.apache.org" <us...@kafka.apache.org>
> >Sent: Wednesday, May 8, 2013 11:20 AM
> >Subject: Re: Kafka Monitoring, 0.7 vs. 0.8 JMX
> >
> >
> >0.8 JMX is different from 0.7. In 0.8, all jmx beans are exposed through
> >metrics. One can attach a metric reporter for monitoring.
> >
> >Thanks,
> >
> >Jun
> >
> >
> >On Tue, May 7, 2013 at 1:45 PM, Otis Gospodnetic <
> otis.gospodnetic@gmail.com
> >> wrote:
> >
> >> Hi,
> >>
> >> We're considering adding Kafka monitoring to SPM (see
> >> http://sematext.com/spm/index.html ). We use Kafka ourselves and
> >> would like to see our Kafka metrics in SPM along with Hadoop, HBase,
> >> Solr, and other metrics we monitor.
> >>
> >> My questions:
> >> * What do people currently use for monitoring Kafka?
> >> * Is there apetite for adding Kafka support to SPM?
> >> * How different is the structure/content of JMX for 0.7 vs. 0.8? Is
> >> 0.8 a backwards-compatible superset of 0.7?
> >>
> >> Thanks,
> >> Otis
> >> --
> >> Search Analytics - http://sematext.com/search-analytics/index.html
> >> Performance Monitoring - http://sematext.com/spm/index.html
> >>
> >
> >
> >
>
Re: Kafka Monitoring, 0.7 vs. 0.8 JMX
Posted by Jun Rao <ju...@gmail.com>.
Otis,
0.8 jmx is quite different from 0.7.
Thanks,
Jun
On Wed, May 8, 2013 at 2:10 PM, Otis Gospodnetic <otis_gospodnetic@yahoo.com
> wrote:
> Hi Jun,
>
> Does that imply that what 0.8 puts in JMX is a superset of what's in JMX
> in 0.7?
> Or have names or types of beans changed?
>
> Also, do you recommend getting metrics via JMX or via HTTP?
>
> Thanks,
> Otis
> ----
> Monitoring for Solr / ElasticSearch / HBase / Hadoop -
> http://sematext.com/spm
>
>
>
>
> >________________________________
> > From: Jun Rao <ju...@gmail.com>
> >To: "users@kafka.apache.org" <us...@kafka.apache.org>
> >Sent: Wednesday, May 8, 2013 11:20 AM
> >Subject: Re: Kafka Monitoring, 0.7 vs. 0.8 JMX
> >
> >
> >0.8 JMX is different from 0.7. In 0.8, all jmx beans are exposed through
> >metrics. One can attach a metric reporter for monitoring.
> >
> >Thanks,
> >
> >Jun
> >
> >
> >On Tue, May 7, 2013 at 1:45 PM, Otis Gospodnetic <
> otis.gospodnetic@gmail.com
> >> wrote:
> >
> >> Hi,
> >>
> >> We're considering adding Kafka monitoring to SPM (see
> >> http://sematext.com/spm/index.html ). We use Kafka ourselves and
> >> would like to see our Kafka metrics in SPM along with Hadoop, HBase,
> >> Solr, and other metrics we monitor.
> >>
> >> My questions:
> >> * What do people currently use for monitoring Kafka?
> >> * Is there apetite for adding Kafka support to SPM?
> >> * How different is the structure/content of JMX for 0.7 vs. 0.8? Is
> >> 0.8 a backwards-compatible superset of 0.7?
> >>
> >> Thanks,
> >> Otis
> >> --
> >> Search Analytics - http://sematext.com/search-analytics/index.html
> >> Performance Monitoring - http://sematext.com/spm/index.html
> >>
> >
> >
> >
>
Re: Kafka Monitoring, 0.7 vs. 0.8 JMX
Posted by Dragos Manolescu <Dr...@servicenow.com>.
>From the JmxReporter section of the metrics manual:
Warning
We don¹t recommend that you try to gather metrics from your production
environment. JMX¹s RPC API is fragile and bonkers. For development
purposes and browsing, though, it can be very useful.
-Dragos
On 5/8/13 2:10 PM, "Otis Gospodnetic" <ot...@yahoo.com> wrote:
>Also, do you recommend getting metrics via JMX or via HTTP?
Re: Kafka Monitoring, 0.7 vs. 0.8 JMX
Posted by Otis Gospodnetic <ot...@yahoo.com>.
Hi Jun,
Does that imply that what 0.8 puts in JMX is a superset of what's in JMX in 0.7?
Or have names or types of beans changed?
Also, do you recommend getting metrics via JMX or via HTTP?
Thanks,
Otis
----
Monitoring for Solr / ElasticSearch / HBase / Hadoop - http://sematext.com/spm
>________________________________
> From: Jun Rao <ju...@gmail.com>
>To: "users@kafka.apache.org" <us...@kafka.apache.org>
>Sent: Wednesday, May 8, 2013 11:20 AM
>Subject: Re: Kafka Monitoring, 0.7 vs. 0.8 JMX
>
>
>0.8 JMX is different from 0.7. In 0.8, all jmx beans are exposed through
>metrics. One can attach a metric reporter for monitoring.
>
>Thanks,
>
>Jun
>
>
>On Tue, May 7, 2013 at 1:45 PM, Otis Gospodnetic <otis.gospodnetic@gmail.com
>> wrote:
>
>> Hi,
>>
>> We're considering adding Kafka monitoring to SPM (see
>> http://sematext.com/spm/index.html ). We use Kafka ourselves and
>> would like to see our Kafka metrics in SPM along with Hadoop, HBase,
>> Solr, and other metrics we monitor.
>>
>> My questions:
>> * What do people currently use for monitoring Kafka?
>> * Is there apetite for adding Kafka support to SPM?
>> * How different is the structure/content of JMX for 0.7 vs. 0.8? Is
>> 0.8 a backwards-compatible superset of 0.7?
>>
>> Thanks,
>> Otis
>> --
>> Search Analytics - http://sematext.com/search-analytics/index.html
>> Performance Monitoring - http://sematext.com/spm/index.html
>>
>
>
>
Re: Kafka Monitoring, 0.7 vs. 0.8 JMX
Posted by Jun Rao <ju...@gmail.com>.
0.8 JMX is different from 0.7. In 0.8, all jmx beans are exposed through
metrics. One can attach a metric reporter for monitoring.
Thanks,
Jun
On Tue, May 7, 2013 at 1:45 PM, Otis Gospodnetic <otis.gospodnetic@gmail.com
> wrote:
> Hi,
>
> We're considering adding Kafka monitoring to SPM (see
> http://sematext.com/spm/index.html ). We use Kafka ourselves and
> would like to see our Kafka metrics in SPM along with Hadoop, HBase,
> Solr, and other metrics we monitor.
>
> My questions:
> * What do people currently use for monitoring Kafka?
> * Is there apetite for adding Kafka support to SPM?
> * How different is the structure/content of JMX for 0.7 vs. 0.8? Is
> 0.8 a backwards-compatible superset of 0.7?
>
> Thanks,
> Otis
> --
> Search Analytics - http://sematext.com/search-analytics/index.html
> Performance Monitoring - http://sematext.com/spm/index.html
>