You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Mark <st...@gmail.com> on 2011/03/20 16:43:26 UTC
Chukwa - Lightweight agents
Sorry but it doesn't look like Chukwa mailing list exists anymore?
Is there an easy way to set up lightweight agents on cluster of machines
instead of downloading the full Chukwa source (+50mb)?
Has anyone build separate RPM's for the agents/collectors?
Thanks
Re: Chukwa - Lightweight agents
Posted by Bill Graham <bi...@gmail.com>.
Yes, we run light weight Chukwa agents and collectors only, using
Chukwa just as you describe. We've been doing so for over a year or so
without many issues. The code is fairly easy to extend when needed. We
rolled our own collector, agent and demux RPMs.
The monitoring peice of chukwa is optional and we don't use that part.
On Sun, Mar 20, 2011 at 6:47 PM, Ted Dunning <td...@maprtech.com> wrote:
> Bummer.
>
> On Sun, Mar 20, 2011 at 10:12 AM, Mark <st...@gmail.com> wrote:
>
>> We tried flume however there are some pretty strange bugs occurring which
>> prevent us from using it.
>>
>>
>> http://groups.google.com/a/cloudera.org/group/flume-user/browse_thread/thread/66c6aecec9d1869b
>>
>>
>> On 3/20/11 10:03 AM, Ted Dunning wrote:
>>
>> OpenTSDB is purely a monitoring solution which is the primary mission of
>> chukwa.
>>
>> If you are looking for data import, what about Flume?
>>
>> On Sun, Mar 20, 2011 at 9:59 AM, Mark <st...@gmail.com> wrote:
>>
>>> Thanks but we need Chukwa to aggregate and store files from across our
>>> app servers into Hadoop. Doesn't really look like opentsdb is meant for
>>> that. I could be wrong though?
>>>
>>>
>>> On 3/20/11 9:49 AM, Ted Dunning wrote:
>>>
>>> Take a look at openTsDb at http://opentsdb.net/
>>>
>>> It provides lots of the capability in a MUCH simpler package.
>>>
>>> On Sun, Mar 20, 2011 at 8:43 AM, Mark <st...@gmail.com> wrote:
>>>
>>>> Sorry but it doesn't look like Chukwa mailing list exists anymore?
>>>>
>>>> Is there an easy way to set up lightweight agents on cluster of machines
>>>> instead of downloading the full Chukwa source (+50mb)?
>>>>
>>>> Has anyone build separate RPM's for the agents/collectors?
>>>>
>>>> Thanks
>>>>
>>>
>>>
>>
>
Re: Chukwa - Lightweight agents
Posted by Bill Graham <bi...@gmail.com>.
Yes, we run light weight Chukwa agents and collectors only, using
Chukwa just as you describe. We've been doing so for over a year or so
without many issues. The code is fairly easy to extend when needed. We
rolled our own collector, agent and demux RPMs.
The monitoring peice of chukwa is optional and we don't use that part.
On Sun, Mar 20, 2011 at 6:47 PM, Ted Dunning <td...@maprtech.com> wrote:
> Bummer.
>
> On Sun, Mar 20, 2011 at 10:12 AM, Mark <st...@gmail.com> wrote:
>
>> We tried flume however there are some pretty strange bugs occurring which
>> prevent us from using it.
>>
>>
>> http://groups.google.com/a/cloudera.org/group/flume-user/browse_thread/thread/66c6aecec9d1869b
>>
>>
>> On 3/20/11 10:03 AM, Ted Dunning wrote:
>>
>> OpenTSDB is purely a monitoring solution which is the primary mission of
>> chukwa.
>>
>> If you are looking for data import, what about Flume?
>>
>> On Sun, Mar 20, 2011 at 9:59 AM, Mark <st...@gmail.com> wrote:
>>
>>> Thanks but we need Chukwa to aggregate and store files from across our
>>> app servers into Hadoop. Doesn't really look like opentsdb is meant for
>>> that. I could be wrong though?
>>>
>>>
>>> On 3/20/11 9:49 AM, Ted Dunning wrote:
>>>
>>> Take a look at openTsDb at http://opentsdb.net/
>>>
>>> It provides lots of the capability in a MUCH simpler package.
>>>
>>> On Sun, Mar 20, 2011 at 8:43 AM, Mark <st...@gmail.com> wrote:
>>>
>>>> Sorry but it doesn't look like Chukwa mailing list exists anymore?
>>>>
>>>> Is there an easy way to set up lightweight agents on cluster of machines
>>>> instead of downloading the full Chukwa source (+50mb)?
>>>>
>>>> Has anyone build separate RPM's for the agents/collectors?
>>>>
>>>> Thanks
>>>>
>>>
>>>
>>
>
Re: Chukwa - Lightweight agents
Posted by Ted Dunning <td...@maprtech.com>.
Bummer.
On Sun, Mar 20, 2011 at 10:12 AM, Mark <st...@gmail.com> wrote:
> We tried flume however there are some pretty strange bugs occurring which
> prevent us from using it.
>
>
> http://groups.google.com/a/cloudera.org/group/flume-user/browse_thread/thread/66c6aecec9d1869b
>
>
> On 3/20/11 10:03 AM, Ted Dunning wrote:
>
> OpenTSDB is purely a monitoring solution which is the primary mission of
> chukwa.
>
> If you are looking for data import, what about Flume?
>
> On Sun, Mar 20, 2011 at 9:59 AM, Mark <st...@gmail.com> wrote:
>
>> Thanks but we need Chukwa to aggregate and store files from across our
>> app servers into Hadoop. Doesn't really look like opentsdb is meant for
>> that. I could be wrong though?
>>
>>
>> On 3/20/11 9:49 AM, Ted Dunning wrote:
>>
>> Take a look at openTsDb at http://opentsdb.net/
>>
>> It provides lots of the capability in a MUCH simpler package.
>>
>> On Sun, Mar 20, 2011 at 8:43 AM, Mark <st...@gmail.com> wrote:
>>
>>> Sorry but it doesn't look like Chukwa mailing list exists anymore?
>>>
>>> Is there an easy way to set up lightweight agents on cluster of machines
>>> instead of downloading the full Chukwa source (+50mb)?
>>>
>>> Has anyone build separate RPM's for the agents/collectors?
>>>
>>> Thanks
>>>
>>
>>
>
Re: Chukwa - Lightweight agents
Posted by Mark <st...@gmail.com>.
We tried flume however there are some pretty strange bugs occurring
which prevent us from using it.
http://groups.google.com/a/cloudera.org/group/flume-user/browse_thread/thread/66c6aecec9d1869b
On 3/20/11 10:03 AM, Ted Dunning wrote:
> OpenTSDB is purely a monitoring solution which is the primary mission
> of chukwa.
>
> If you are looking for data import, what about Flume?
>
> On Sun, Mar 20, 2011 at 9:59 AM, Mark <static.void.dev@gmail.com
> <ma...@gmail.com>> wrote:
>
> Thanks but we need Chukwa to aggregate and store files from across
> our app servers into Hadoop. Doesn't really look like opentsdb is
> meant for that. I could be wrong though?
>
>
> On 3/20/11 9:49 AM, Ted Dunning wrote:
>> Take a look at openTsDb at http://opentsdb.net/
>>
>> It provides lots of the capability in a MUCH simpler package.
>>
>> On Sun, Mar 20, 2011 at 8:43 AM, Mark <static.void.dev@gmail.com
>> <ma...@gmail.com>> wrote:
>>
>> Sorry but it doesn't look like Chukwa mailing list exists
>> anymore?
>>
>> Is there an easy way to set up lightweight agents on cluster
>> of machines instead of downloading the full Chukwa source
>> (+50mb)?
>>
>> Has anyone build separate RPM's for the agents/collectors?
>>
>> Thanks
>>
>>
>
Re: Chukwa - Lightweight agents
Posted by Ted Dunning <td...@maprtech.com>.
OpenTSDB is purely a monitoring solution which is the primary mission of
chukwa.
If you are looking for data import, what about Flume?
On Sun, Mar 20, 2011 at 9:59 AM, Mark <st...@gmail.com> wrote:
> Thanks but we need Chukwa to aggregate and store files from across our app
> servers into Hadoop. Doesn't really look like opentsdb is meant for that. I
> could be wrong though?
>
>
> On 3/20/11 9:49 AM, Ted Dunning wrote:
>
> Take a look at openTsDb at http://opentsdb.net/
>
> It provides lots of the capability in a MUCH simpler package.
>
> On Sun, Mar 20, 2011 at 8:43 AM, Mark <st...@gmail.com> wrote:
>
>> Sorry but it doesn't look like Chukwa mailing list exists anymore?
>>
>> Is there an easy way to set up lightweight agents on cluster of machines
>> instead of downloading the full Chukwa source (+50mb)?
>>
>> Has anyone build separate RPM's for the agents/collectors?
>>
>> Thanks
>>
>
>
Re: Chukwa - Lightweight agents
Posted by Mark <st...@gmail.com>.
Thanks but we need Chukwa to aggregate and store files from across our
app servers into Hadoop. Doesn't really look like opentsdb is meant for
that. I could be wrong though?
On 3/20/11 9:49 AM, Ted Dunning wrote:
> Take a look at openTsDb at http://opentsdb.net/
>
> It provides lots of the capability in a MUCH simpler package.
>
> On Sun, Mar 20, 2011 at 8:43 AM, Mark <static.void.dev@gmail.com
> <ma...@gmail.com>> wrote:
>
> Sorry but it doesn't look like Chukwa mailing list exists anymore?
>
> Is there an easy way to set up lightweight agents on cluster of
> machines instead of downloading the full Chukwa source (+50mb)?
>
> Has anyone build separate RPM's for the agents/collectors?
>
> Thanks
>
>
Re: Chukwa - Lightweight agents
Posted by Ted Dunning <td...@maprtech.com>.
Take a look at openTsDb at http://opentsdb.net/
It provides lots of the capability in a MUCH simpler package.
On Sun, Mar 20, 2011 at 8:43 AM, Mark <st...@gmail.com> wrote:
> Sorry but it doesn't look like Chukwa mailing list exists anymore?
>
> Is there an easy way to set up lightweight agents on cluster of machines
> instead of downloading the full Chukwa source (+50mb)?
>
> Has anyone build separate RPM's for the agents/collectors?
>
> Thanks
>