You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@chukwa.apache.org by Jerome Boulon <jb...@netflix.com> on 2010/02/24 18:41:40 UTC

Survey - Chukwa usage

Hi,
I would like to get some stats on how Chukwa is used?
Thanks in advance,
  /Jerome.

* cluster status 
> * personal 
> * business ­ experiment
> * business ­ dev 
> * business ­ test
> * business ­ prod

* cluster size
> * < 10 nodes 
> * < 100 nodes 
> * < 200 nodes 
> * < 500 nodes 
> * > 1000 nodes

* Chukwa usage
> * monitoring 
> * analytics

* Are you using Chukwa to collect:
> * Log files 
> * Metrics information

* Cluster running
> * in your own data center
> * in the cloud (ex: amazon ec2)

* which Chukwa components are you using?
> * UI (HICC) 
> * Log4j Appender 
> * Agent adaptors (list)
> * Archiver 
> * Demuxer

* Data analytics on ChukwaRecords
> * pig 
> * M/R

* Would you be interested in an agent less solution?

* Would you like to be able to use Hive instead of/in addition to Pig?

* What from your point of view is missing in Chukwa, today?



Re: Survey - Chukwa usage

Posted by James Seigel <ja...@tynt.com>.
On 2010-02-24, at 10:41 AM, Jerome Boulon wrote:

cluster status 
business – prod

cluster size
< 100 nodes

Chukwa usage
analytics

Are you using Chukwa to collect:
Log files

Cluster running
in your own data center

which Chukwa components are you using?
Agent adaptors (list)(DirTailingAdaptor)
Archiver
Demuxer

Data analytics on ChukwaRecords
pig

Would you be interested in an agent less solution? 
		no

Would you like to be able to use Hive instead of/in addition to Pig?
		in addition

What from your point of view is missing in Chukwa, today?
	Better docs on data flow.  Better plugability. Triggers



Re: Survey - Chukwa usage

Posted by Guillermo Pérez <bi...@tuenti.com>.
cluster status: business – dev
cluster size: < 10 nodes (for now)
Chukwa usage: analytics
Are you using Chukwa to collect: Custom application events (more like a log)
Cluster running: in your own data center
which Chukwa components are you using? UI (HICC), Archiver, Demuxer, BackFill
Data analytics on ChukwaRecords: pig
Would you be interested in an agent less solution? Yes. Integration
with syslog-ng
Would you like to be able to use Hive instead of/in addition to Pig? Yes

What from your point of view is missing in Chukwa, today?
- Better reports of errors during demux.
- More integrated and easy to modify framework for generating reports
and push them to mysql. HICC does this, but is hard adapt to your
needs.
-- 
Guille -ℬḭṩḩø- <bi...@tuenti.com>
:wq

Re: Survey - Chukwa usage

Posted by Eric Yang <ey...@yahoo-inc.com>.
Cluster Status ­ business dev
Cluster Size ­ 10 nodes
Chukwa Usage: monitoring + analytics
Cluster: yahoo data center
All components
Data analytics: pig+MR
Agent less solution: no preference
Hive: no preference

What is missing:
- Missing good storage format.
- Missing good post demux report generator.
- Missing good data retention process.
- Missing full body index for Archive for search, i.e. find log entries.
- The list goes on and on...

It will take time to build all those features.

Regards,
Eric

On 2/24/10 9:41 AM, "Jerome Boulon" <jb...@netflix.com> wrote:

> Hi,
> I would like to get some stats on how Chukwa is used?
> Thanks in advance,
>   /Jerome.
> 
> * cluster status 
>> * personal 
>> * business ­ experiment
>> * business ­ dev
>> * business ­ test
>> * business ­ prod
> 
> * cluster size
>> * < 10 nodes 
>> * < 100 nodes 
>> * < 200 nodes 
>> * < 500 nodes 
>> * > 1000 nodes
> 
> * Chukwa usage
>> * monitoring 
>> * analytics
> 
> * Are you using Chukwa to collect:
>> * Log files 
>> * Metrics information
> 
> * Cluster running
>> * in your own data center
>> * in the cloud (ex: amazon ec2)
> 
> * which Chukwa components are you using?
>> * UI (HICC) 
>> * Log4j Appender
>> * Agent adaptors (list)
>> * Archiver 
>> * Demuxer
> 
> * Data analytics on ChukwaRecords
>> * pig 
>> * M/R
> 
> * Would you be interested in an agent less solution?
> 
> * Would you like to be able to use Hive instead of/in addition to Pig?
> 
> * What from your point of view is missing in Chukwa, today?
> 
>