You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by "chidigam ." <bh...@gmail.com> on 2017/11/09 01:10:05 UTC

Kafka Monitoring..

Hi All,
What is the simplest way of monitoring the metrics in kaka brokers?
Is there any opensource available?
Any help in this regards is appreciated.

Regards
Bhanu

Re: Kafka Monitoring..

Posted by Ted Yu <yu...@gmail.com>.
Have you seen this thread ?

http://search-hadoop.com/m/Kafka/uyzND1Q6wyNBj42g?subj=Kafka+Monitoring

On Wed, Nov 8, 2017 at 5:10 PM, chidigam . <bh...@gmail.com> wrote:

> Hi All,
> What is the simplest way of monitoring the metrics in kaka brokers?
> Is there any opensource available?
> Any help in this regards is appreciated.
>
> Regards
> Bhanu
>

Re: Kafka Monitoring..

Posted by David Garcia <da...@spiceworks.com>.
Yeah…jmx is really the cheap/easy way to monitor kafka.  You should also monitor OS metrics like pageScanD and pageScanK.  Those were very helpful for us.

-David

On 11/9/17, 11:40 AM, "Andrew Otto" <ot...@wikimedia.org> wrote:

    We’ve recently started using Prometheus, and use Prometheus JMX Exporter
    <https://github.com/prometheus/jmx_exporter> to get Kafka metrics into
    prometheus.  Here’s our JMX Exporter config:
    https://github.com/wikimedia/puppet/blob/production/modules/profile/files/kafka/broker_prometheus_jmx_exporter.yaml
    
    
    
    On Thu, Nov 9, 2017 at 11:42 AM, David Garcia <da...@spiceworks.com> wrote:
    
    > JMX was pretty easy to setup for us.  Look up the various jmx beans
    > locally for your particular version of kafka (i.e. with jconsole..etc)
    >
    > https://github.com/jmxtrans/jmxtrans
    >
    > On 11/8/17, 7:10 PM, "chidigam ." <bh...@gmail.com> wrote:
    >
    >     Hi All,
    >     What is the simplest way of monitoring the metrics in kaka brokers?
    >     Is there any opensource available?
    >     Any help in this regards is appreciated.
    >
    >     Regards
    >     Bhanu
    >
    >
    >

Re: Kafka Monitoring..

Posted by Andrew Otto <ot...@wikimedia.org>.
We’ve recently started using Prometheus, and use Prometheus JMX Exporter
<https://github.com/prometheus/jmx_exporter> to get Kafka metrics into
prometheus.  Here’s our JMX Exporter config:
https://github.com/wikimedia/puppet/blob/production/modules/profile/files/kafka/broker_prometheus_jmx_exporter.yaml



On Thu, Nov 9, 2017 at 11:42 AM, David Garcia <da...@spiceworks.com> wrote:

> JMX was pretty easy to setup for us.  Look up the various jmx beans
> locally for your particular version of kafka (i.e. with jconsole..etc)
>
> https://github.com/jmxtrans/jmxtrans
>
> On 11/8/17, 7:10 PM, "chidigam ." <bh...@gmail.com> wrote:
>
>     Hi All,
>     What is the simplest way of monitoring the metrics in kaka brokers?
>     Is there any opensource available?
>     Any help in this regards is appreciated.
>
>     Regards
>     Bhanu
>
>
>

Re: Kafka Monitoring..

Posted by David Garcia <da...@spiceworks.com>.
JMX was pretty easy to setup for us.  Look up the various jmx beans locally for your particular version of kafka (i.e. with jconsole..etc)

https://github.com/jmxtrans/jmxtrans

On 11/8/17, 7:10 PM, "chidigam ." <bh...@gmail.com> wrote:

    Hi All,
    What is the simplest way of monitoring the metrics in kaka brokers?
    Is there any opensource available?
    Any help in this regards is appreciated.
    
    Regards
    Bhanu