You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flume.apache.org by Jonathan Hsieh <jo...@cloudera.com> on 2011/07/26 03:02:33 UTC
Starting to document metrics monitoring system in wiki.
Hi all,
I've started adding documentation on the current metrics monitoring system
built into Flume Nodes and Flume Masters [1] . We'd really like some
feedback on what metrics are provided and if the granularity is about
right, and if it is adequate for integrating with monitoring/alerting
systems like nagios/muni/ganglia/zenoss/et al. Once this gets solidified,
we'll fold it into the formal documentation.
If folks are already using this, I'd love to hear about blog posts or
recipes!
Thanks,
Jon.
[1] https://cwiki.apache.org/confluence/display/FLUME/Monitoring+Flume
--
// Jonathan Hsieh (shay)
// Software Engineer, Cloudera
// jon@cloudera.com
Re: Starting to document metrics monitoring system in wiki.
Posted by Eric Sammer <es...@cloudera.com>.
REST and JSON APIs already exist (in a deviously undocumented form).
Not sure about JSONP although I don't see a reason not to do as you
ask. File a JIRA maybe?
On Mon, Jul 25, 2011 at 7:34 PM, Dan Everton <da...@iocaine.org> wrote:
> On Mon, 25 Jul 2011 19:14 -0700, "Eric Sammer" <es...@cloudera.com>
> wrote:
>> Mike:
>>
>> Are four letter words preferable to, say, REST? I think REST is nicer
>> (maybe with 4LW URLs and responses rather than JSON) from an
>> accessibility point of view. I don't see a reason we couldn't do it,
>
> If you're going to go REST and JSON please make sure you support JSONP
> for the output. Same origin policy makes building web based dashboards
> difficult without this.
>
> Jersey supports this if you use the JSONWithPadding response object.
>
> Cheers,
> Dan
>
>
--
Eric Sammer
twitter: esammer
data: www.cloudera.com
Re: Starting to document metrics monitoring system in wiki.
Posted by Eric Sammer <es...@cloudera.com>.
REST and JSON APIs already exist (in a deviously undocumented form).
Not sure about JSONP although I don't see a reason not to do as you
ask. File a JIRA maybe?
On Mon, Jul 25, 2011 at 7:34 PM, Dan Everton <da...@iocaine.org> wrote:
> On Mon, 25 Jul 2011 19:14 -0700, "Eric Sammer" <es...@cloudera.com>
> wrote:
>> Mike:
>>
>> Are four letter words preferable to, say, REST? I think REST is nicer
>> (maybe with 4LW URLs and responses rather than JSON) from an
>> accessibility point of view. I don't see a reason we couldn't do it,
>
> If you're going to go REST and JSON please make sure you support JSONP
> for the output. Same origin policy makes building web based dashboards
> difficult without this.
>
> Jersey supports this if you use the JSONWithPadding response object.
>
> Cheers,
> Dan
>
>
--
Eric Sammer
twitter: esammer
data: www.cloudera.com
Re: Starting to document metrics monitoring system in wiki.
Posted by Dan Everton <da...@iocaine.org>.
On Mon, 25 Jul 2011 19:14 -0700, "Eric Sammer" <es...@cloudera.com>
wrote:
> Mike:
>
> Are four letter words preferable to, say, REST? I think REST is nicer
> (maybe with 4LW URLs and responses rather than JSON) from an
> accessibility point of view. I don't see a reason we couldn't do it,
If you're going to go REST and JSON please make sure you support JSONP
for the output. Same origin policy makes building web based dashboards
difficult without this.
Jersey supports this if you use the JSONWithPadding response object.
Cheers,
Dan
Re: Starting to document metrics monitoring system in wiki.
Posted by Dan Everton <da...@iocaine.org>.
On Mon, 25 Jul 2011 19:14 -0700, "Eric Sammer" <es...@cloudera.com>
wrote:
> Mike:
>
> Are four letter words preferable to, say, REST? I think REST is nicer
> (maybe with 4LW URLs and responses rather than JSON) from an
> accessibility point of view. I don't see a reason we couldn't do it,
If you're going to go REST and JSON please make sure you support JSONP
for the output. Same origin policy makes building web based dashboards
difficult without this.
Jersey supports this if you use the JSONWithPadding response object.
Cheers,
Dan
Re: Starting to document metrics monitoring system in wiki.
Posted by Eric Sammer <es...@cloudera.com>.
Mike:
Are four letter words preferable to, say, REST? I think REST is nicer
(maybe with 4LW URLs and responses rather than JSON) from an
accessibility point of view. I don't see a reason we couldn't do it,
but it would be *great* to back away from the "everything by the
(kitchen?) sink" mindset in terms of features.
(I demand double lucky number one points for my sink pun. Yes, you're
welcome. ;))
Thanks!
On Mon, Jul 25, 2011 at 6:10 PM, Mike <mi...@gmail.com> wrote:
> Does Flume support the "four letter words" style of monitoring,
> similar to what Apache Zookeeper has?
>
> Also, from the Master perspective, I'd like to see the "Node status"
> section in the output - I don't know if the config and mapping are as
> important to an automated monitoring system.
>
> Moving forward, when multi-master is implemented in a more predictable
> fashion, if one of the outputs would contain the list of masters, as
> well whether the current master is the "active" master.
>
> 4-letter words, refer to:
> http://phunt1.wordpress.com/2010/03/29/monitoring-zookeeper-3-3-even-more-cussin/
> https://issues.apache.org/jira/browse/ZOOKEEPER-744
>
> On Mon, Jul 25, 2011 at 9:02 PM, Jonathan Hsieh <jo...@cloudera.com> wrote:
>> Hi all,
>> I've started adding documentation on the current metrics monitoring system
>> built into Flume Nodes and Flume Masters [1] . We'd really like some
>> feedback on what metrics are provided and if the granularity is about
>> right, and if it is adequate for integrating with monitoring/alerting
>> systems like nagios/muni/ganglia/zenoss/et al. Once this gets solidified,
>> we'll fold it into the formal documentation.
>> If folks are already using this, I'd love to hear about blog posts or
>> recipes!
>> Thanks,
>> Jon.
>> [1] https://cwiki.apache.org/confluence/display/FLUME/Monitoring+Flume
>>
>> --
>> // Jonathan Hsieh (shay)
>> // Software Engineer, Cloudera
>> // jon@cloudera.com
>>
>>
>
--
Eric Sammer
twitter: esammer
data: www.cloudera.com
Re: Starting to document metrics monitoring system in wiki.
Posted by Eric Sammer <es...@cloudera.com>.
Mike:
Are four letter words preferable to, say, REST? I think REST is nicer
(maybe with 4LW URLs and responses rather than JSON) from an
accessibility point of view. I don't see a reason we couldn't do it,
but it would be *great* to back away from the "everything by the
(kitchen?) sink" mindset in terms of features.
(I demand double lucky number one points for my sink pun. Yes, you're
welcome. ;))
Thanks!
On Mon, Jul 25, 2011 at 6:10 PM, Mike <mi...@gmail.com> wrote:
> Does Flume support the "four letter words" style of monitoring,
> similar to what Apache Zookeeper has?
>
> Also, from the Master perspective, I'd like to see the "Node status"
> section in the output - I don't know if the config and mapping are as
> important to an automated monitoring system.
>
> Moving forward, when multi-master is implemented in a more predictable
> fashion, if one of the outputs would contain the list of masters, as
> well whether the current master is the "active" master.
>
> 4-letter words, refer to:
> http://phunt1.wordpress.com/2010/03/29/monitoring-zookeeper-3-3-even-more-cussin/
> https://issues.apache.org/jira/browse/ZOOKEEPER-744
>
> On Mon, Jul 25, 2011 at 9:02 PM, Jonathan Hsieh <jo...@cloudera.com> wrote:
>> Hi all,
>> I've started adding documentation on the current metrics monitoring system
>> built into Flume Nodes and Flume Masters [1] . We'd really like some
>> feedback on what metrics are provided and if the granularity is about
>> right, and if it is adequate for integrating with monitoring/alerting
>> systems like nagios/muni/ganglia/zenoss/et al. Once this gets solidified,
>> we'll fold it into the formal documentation.
>> If folks are already using this, I'd love to hear about blog posts or
>> recipes!
>> Thanks,
>> Jon.
>> [1] https://cwiki.apache.org/confluence/display/FLUME/Monitoring+Flume
>>
>> --
>> // Jonathan Hsieh (shay)
>> // Software Engineer, Cloudera
>> // jon@cloudera.com
>>
>>
>
--
Eric Sammer
twitter: esammer
data: www.cloudera.com
Re: Starting to document metrics monitoring system in wiki.
Posted by Mike <mi...@gmail.com>.
Does Flume support the "four letter words" style of monitoring,
similar to what Apache Zookeeper has?
Also, from the Master perspective, I'd like to see the "Node status"
section in the output - I don't know if the config and mapping are as
important to an automated monitoring system.
Moving forward, when multi-master is implemented in a more predictable
fashion, if one of the outputs would contain the list of masters, as
well whether the current master is the "active" master.
4-letter words, refer to:
http://phunt1.wordpress.com/2010/03/29/monitoring-zookeeper-3-3-even-more-cussin/
https://issues.apache.org/jira/browse/ZOOKEEPER-744
On Mon, Jul 25, 2011 at 9:02 PM, Jonathan Hsieh <jo...@cloudera.com> wrote:
> Hi all,
> I've started adding documentation on the current metrics monitoring system
> built into Flume Nodes and Flume Masters [1] . We'd really like some
> feedback on what metrics are provided and if the granularity is about
> right, and if it is adequate for integrating with monitoring/alerting
> systems like nagios/muni/ganglia/zenoss/et al. Once this gets solidified,
> we'll fold it into the formal documentation.
> If folks are already using this, I'd love to hear about blog posts or
> recipes!
> Thanks,
> Jon.
> [1] https://cwiki.apache.org/confluence/display/FLUME/Monitoring+Flume
>
> --
> // Jonathan Hsieh (shay)
> // Software Engineer, Cloudera
> // jon@cloudera.com
>
>
Re: Starting to document metrics monitoring system in wiki.
Posted by Mike <mi...@gmail.com>.
Does Flume support the "four letter words" style of monitoring,
similar to what Apache Zookeeper has?
Also, from the Master perspective, I'd like to see the "Node status"
section in the output - I don't know if the config and mapping are as
important to an automated monitoring system.
Moving forward, when multi-master is implemented in a more predictable
fashion, if one of the outputs would contain the list of masters, as
well whether the current master is the "active" master.
4-letter words, refer to:
http://phunt1.wordpress.com/2010/03/29/monitoring-zookeeper-3-3-even-more-cussin/
https://issues.apache.org/jira/browse/ZOOKEEPER-744
On Mon, Jul 25, 2011 at 9:02 PM, Jonathan Hsieh <jo...@cloudera.com> wrote:
> Hi all,
> I've started adding documentation on the current metrics monitoring system
> built into Flume Nodes and Flume Masters [1] . We'd really like some
> feedback on what metrics are provided and if the granularity is about
> right, and if it is adequate for integrating with monitoring/alerting
> systems like nagios/muni/ganglia/zenoss/et al. Once this gets solidified,
> we'll fold it into the formal documentation.
> If folks are already using this, I'd love to hear about blog posts or
> recipes!
> Thanks,
> Jon.
> [1] https://cwiki.apache.org/confluence/display/FLUME/Monitoring+Flume
>
> --
> // Jonathan Hsieh (shay)
> // Software Engineer, Cloudera
> // jon@cloudera.com
>
>