You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flume.apache.org by Zhishan Li <zh...@gmail.com> on 2015/11/19 10:11:34 UTC
How to monitor flume agents
Is there a way to simply and conveniently monitor flume agents?
Currently, I check the flume’s log on each flume node everyday.
Thanks,
Re: How to monitor flume agents
Posted by Sharninder <sh...@gmail.com>.
Monitoring requirements are different for different teams but have you looked at collectd? That with graphite/grafana could work for you.
> On 20-Nov-2015, at 10:00 AM, Hemanth Abbina <He...@eiqnetworks.com> wrote:
>
> Just a follow up question.
>
> We need a remote Linux server monitoring tool, to monitor our cluster resources. Parameters to monitor are CPU, Memory, Disk space, IO read/write, Network etc.
>
> Is there any lightweight tool for that ?
>
> We have examined Nagios for that, but seems to be heavy & complex with lot of features that we don’t need.
>
> -----Original Message-----
> From: Ashish [mailto:paliwalashish@gmail.com]
> Sent: Thursday, November 19, 2015 8:29 PM
> To: user@flume.apache.org
> Subject: Re: How to monitor flume agents
>
> Details here
>
> http://flume.apache.org/FlumeUserGuide.html#monitoring
>
>> On Thu, Nov 19, 2015 at 1:11 AM, Zhishan Li <zh...@gmail.com> wrote:
>>
>>
>> Is there a way to simply and conveniently monitor flume agents?
>>
>> Currently, I check the flume’s log on each flume node everyday.
>>
>>
>> Thanks,
>
>
>
> --
> thanks
> ashish
>
> Blog: http://www.ashishpaliwal.com/blog
> My Photo Galleries: http://www.pbase.com/ashishpaliwal
RE: How to monitor flume agents
Posted by Hemanth Abbina <He...@eiqnetworks.com>.
Just a follow up question.
We need a remote Linux server monitoring tool, to monitor our cluster resources. Parameters to monitor are CPU, Memory, Disk space, IO read/write, Network etc.
Is there any lightweight tool for that ?
We have examined Nagios for that, but seems to be heavy & complex with lot of features that we don’t need.
-----Original Message-----
From: Ashish [mailto:paliwalashish@gmail.com]
Sent: Thursday, November 19, 2015 8:29 PM
To: user@flume.apache.org
Subject: Re: How to monitor flume agents
Details here
http://flume.apache.org/FlumeUserGuide.html#monitoring
On Thu, Nov 19, 2015 at 1:11 AM, Zhishan Li <zh...@gmail.com> wrote:
>
>
> Is there a way to simply and conveniently monitor flume agents?
>
> Currently, I check the flume’s log on each flume node everyday.
>
>
> Thanks,
>
--
thanks
ashish
Blog: http://www.ashishpaliwal.com/blog
My Photo Galleries: http://www.pbase.com/ashishpaliwal
Re: How to monitor flume agents
Posted by Ashish <pa...@gmail.com>.
Details here
http://flume.apache.org/FlumeUserGuide.html#monitoring
On Thu, Nov 19, 2015 at 1:11 AM, Zhishan Li <zh...@gmail.com> wrote:
>
>
> Is there a way to simply and conveniently monitor flume agents?
>
> Currently, I check the flume’s log on each flume node everyday.
>
>
> Thanks,
>
--
thanks
ashish
Blog: http://www.ashishpaliwal.com/blog
My Photo Galleries: http://www.pbase.com/ashishpaliwal
RE: How to monitor flume agents
Posted by Keith Wiggins <ke...@genband.com>.
Hi,
You can set up ‘http’ monitoring when you start your process to get the metrics of your Flume.
Example;
-Dflume.monitoring.type=http -Dflume.monitoring.port=34545
Then you can connect a browser to your IP and port. I use Firefox to do that as the output is JSON. You just need to refresh your browser. Not sure if that’s what you are looking for.
Example;
{"CHANNEL.fileChannel":{"EventPutSuccessCount":"56665000","ChannelFillPercentage":"0.0","Type":"CHANNEL","StopTime":"0","EventPutAttemptCount":"56665000","ChannelSize":"0","StartTime":"1447855497117","EventTakeSuccessCount":"56665000","ChannelCapacity":"1000000","EventTakeAttemptCount":"56719994"},"SOURCE.LSMetrics":{"OpenConnectionCount":"1","Type":"SOURCE","AppendBatchAcceptedCount":"566650","AppendBatchReceivedCount":"566650","EventAcceptedCount":"56665000","AppendReceivedCount":"0","StopTime":"0","StartTime":"1447855497515","EventReceivedCount":"56665000","AppendAcceptedCount":"0"}}
Keith
From: Gonzalo Herreros [mailto:gherreros@gmail.com]
Sent: Thursday, November 19, 2015 4:25 AM
To: user
Subject: Re: How to monitor flume agents
Using a distribution like Cloudera or Hortonworks. They both have free versions.
Alternatively you can use standard linux process monitoring tools.
Regards,
Gonzalo
On 19 November 2015 at 09:11, Zhishan Li <zh...@gmail.com>> wrote:
Is there a way to simply and conveniently monitor flume agents?
Currently, I check the flume’s log on each flume node everyday.
Thanks,
Re: How to monitor flume agents
Posted by Gonzalo Herreros <gh...@gmail.com>.
Using a distribution like Cloudera or Hortonworks. They both have free
versions.
Alternatively you can use standard linux process monitoring tools.
Regards,
Gonzalo
On 19 November 2015 at 09:11, Zhishan Li <zh...@gmail.com> wrote:
>
>
> Is there a way to simply and conveniently monitor flume agents?
>
> Currently, I check the flume’s log on each flume node everyday.
>
>
> Thanks,
>
>