You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@kafka.apache.org by Joe Stein <jo...@stealth.ly> on 2014/04/18 18:51:05 UTC

Reporting Metrics to Apache Kafka and Monitoring with Consumers

Hi, we started a new github project for using Kafka as the central point
for all application and infrastructure metrics
https://github.com/stealthly/metrics-kafka/.

We started off with implementing a Metrics Reporter (for Coda Hale's
metrics) which produces (reports) the statistics to a Kafka topic.  We also
created an example consumer for monitoring those metrics using Riemann.
 More details in the blog post
http://allthingshadoop.com/2014/04/18/metrics-kafka/ and more producers to
follow (probably consumers too).

Thoughts, comments, questions, feedback, tomatoes, pull requests are all
welcome =8^)

/*******************************************
 Joe Stein
 Founder, Principal Consultant
 Big Data Open Source Security LLC
 http://www.stealth.ly
 Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
********************************************/

Re: Reporting Metrics to Apache Kafka and Monitoring with Consumers

Posted by Joe Stein <jo...@stealth.ly>.

Thanks Jun! I added it to the ecosystem page.

On Fri, Apr 18, 2014 at 1:26 PM, Jun Rao <ju...@gmail.com> wrote:

> Wow, Joe. That looks great. Could you add it to our wiki?
>
> Thanks,
>
> Jun
>
>
> On Fri, Apr 18, 2014 at 9:51 AM, Joe Stein <jo...@stealth.ly> wrote:
>
> > Hi, we started a new github project for using Kafka as the central point
> > for all application and infrastructure metrics
> > https://github.com/stealthly/metrics-kafka/.
> >
> > We started off with implementing a Metrics Reporter (for Coda Hale's
> > metrics) which produces (reports) the statistics to a Kafka topic.  We
> also
> > created an example consumer for monitoring those metrics using Riemann.
> >  More details in the blog post
> > http://allthingshadoop.com/2014/04/18/metrics-kafka/ and more producers
> to
> > follow (probably consumers too).
> >
> > Thoughts, comments, questions, feedback, tomatoes, pull requests are all
> > welcome =8^)
> >
> > /*******************************************
> >  Joe Stein
> >  Founder, Principal Consultant
> >  Big Data Open Source Security LLC
> >  http://www.stealth.ly
> >  Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
> > ********************************************/
> >
>

Re: Reporting Metrics to Apache Kafka and Monitoring with Consumers

Posted by Jun Rao <ju...@gmail.com>.

Wow, Joe. That looks great. Could you add it to our wiki?

Thanks,

Jun


On Fri, Apr 18, 2014 at 9:51 AM, Joe Stein <jo...@stealth.ly> wrote:

> Hi, we started a new github project for using Kafka as the central point
> for all application and infrastructure metrics
> https://github.com/stealthly/metrics-kafka/.
>
> We started off with implementing a Metrics Reporter (for Coda Hale's
> metrics) which produces (reports) the statistics to a Kafka topic.  We also
> created an example consumer for monitoring those metrics using Riemann.
>  More details in the blog post
> http://allthingshadoop.com/2014/04/18/metrics-kafka/ and more producers to
> follow (probably consumers too).
>
> Thoughts, comments, questions, feedback, tomatoes, pull requests are all
> welcome =8^)
>
> /*******************************************
>  Joe Stein
>  Founder, Principal Consultant
>  Big Data Open Source Security LLC
>  http://www.stealth.ly
>  Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
> ********************************************/
>