You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by M Singh <ma...@yahoo.com> on 2016/01/01 17:25:59 UTC
Nifi Processor/Connection stats custom window duration
Hi:
The Nifi processors and components show 5 min status in the UI (In, Read/Write, Out, and Tasks/Time) and also in the component stats dialog box.
I wanted to find out if there is a way to 1. Customize this time-window, 2. Provide options for multiple windows (5 minute, 1 hour, 1 day/etc). 3. Compute these dynamically (ie, set the default to 5 minutes, but then change a component configuration to see the stats for the previously processed events by an hour/etc).
I looked in the admin guide (https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html) but could not find a property for setting the stats window duration.
If there is an option to set the stats window duration, please let me know.
Thanks
Mans
Re: Nifi Processor/Connection stats custom window duration
Posted by M Singh <ma...@yahoo.com>.
Hi Joe:
I found the reference to the rest api.
Thanks
Mans
On Friday, January 1, 2016 10:48 AM, M Singh <ma...@yahoo.com> wrote:
Hi Joe:
Can you please elaborate/point me to the where I can find more info about pulling stats via rest api ?
Thanks
On Friday, January 1, 2016 9:52 AM, Joe Witt <jo...@gmail.com> wrote:
Mans
The stats shown on the component are fixed at a 5 minute window. The
request to be able to alter that does come up from time to time. The
idea there is that from a user experience perspective those numbers
are the most relevant to show as a recurring value because over time
you become familiar with them and core more readily determine if
things seem about right. For longer time windows you can right-click
on components to view their stats. Presently that is only retained
for up to 24 hours by default but those settings are configurable in
nifi.properties. Those settings control both how often and for how
long those stats are retained. The framework is designed already to
support plugging into a persistent storage for those stats so it is
just a matter of time before a contrib takes care of that. Also, keep
in mind we expect these statistics to be exported to purpose-built
monitoring systems like Ganglia, Ambari, or others such as Riemann
which Ricky Saltzer is working on. There are two ways to export these
stats. First, is the reporting task API which is an extension point
in NiFi. You can push these stats to an external system using that
mechanism. You can also pull the stats via the REST API.
That said, there is work underway for modernizing the look and feel of
NiFi and it is a great time to talk about what would be most helpful
to see in the UI regarding live/historical stats so please feel free
to send drawings/ideas you have on that.
Thanks
Joe
On Fri, Jan 1, 2016 at 11:25 AM, M Singh <ma...@yahoo.com> wrote:
> Hi:
>
> The Nifi processors and components show 5 min status in the UI (In,
> Read/Write, Out, and Tasks/Time) and also in the component stats dialog box.
>
> I wanted to find out if there is a way to
> 1. Customize this time-window,
> 2. Provide options for multiple windows (5 minute, 1 hour, 1 day/etc).
> 3. Compute these dynamically (ie, set the default to 5 minutes, but then
> change a component configuration to see the stats for the previously
> processed events by an hour/etc).
>
> I looked in the admin guide
> (https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html) but
> could not find a property for setting the stats window duration.
>
> If there is an option to set the stats window duration, please let me know.
>
> Thanks
>
> Mans
>
>
Re: Nifi Processor/Connection stats custom window duration
Posted by M Singh <ma...@yahoo.com>.
Hi Joe:
Can you please elaborate/point me to the where I can find more info about pulling stats via rest api ?
Thanks
On Friday, January 1, 2016 9:52 AM, Joe Witt <jo...@gmail.com> wrote:
Mans
The stats shown on the component are fixed at a 5 minute window. The
request to be able to alter that does come up from time to time. The
idea there is that from a user experience perspective those numbers
are the most relevant to show as a recurring value because over time
you become familiar with them and core more readily determine if
things seem about right. For longer time windows you can right-click
on components to view their stats. Presently that is only retained
for up to 24 hours by default but those settings are configurable in
nifi.properties. Those settings control both how often and for how
long those stats are retained. The framework is designed already to
support plugging into a persistent storage for those stats so it is
just a matter of time before a contrib takes care of that. Also, keep
in mind we expect these statistics to be exported to purpose-built
monitoring systems like Ganglia, Ambari, or others such as Riemann
which Ricky Saltzer is working on. There are two ways to export these
stats. First, is the reporting task API which is an extension point
in NiFi. You can push these stats to an external system using that
mechanism. You can also pull the stats via the REST API.
That said, there is work underway for modernizing the look and feel of
NiFi and it is a great time to talk about what would be most helpful
to see in the UI regarding live/historical stats so please feel free
to send drawings/ideas you have on that.
Thanks
Joe
On Fri, Jan 1, 2016 at 11:25 AM, M Singh <ma...@yahoo.com> wrote:
> Hi:
>
> The Nifi processors and components show 5 min status in the UI (In,
> Read/Write, Out, and Tasks/Time) and also in the component stats dialog box.
>
> I wanted to find out if there is a way to
> 1. Customize this time-window,
> 2. Provide options for multiple windows (5 minute, 1 hour, 1 day/etc).
> 3. Compute these dynamically (ie, set the default to 5 minutes, but then
> change a component configuration to see the stats for the previously
> processed events by an hour/etc).
>
> I looked in the admin guide
> (https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html) but
> could not find a property for setting the stats window duration.
>
> If there is an option to set the stats window duration, please let me know.
>
> Thanks
>
> Mans
>
>
Re: Nifi Processor/Connection stats custom window duration
Posted by Joe Witt <jo...@gmail.com>.
Mans
The stats shown on the component are fixed at a 5 minute window. The
request to be able to alter that does come up from time to time. The
idea there is that from a user experience perspective those numbers
are the most relevant to show as a recurring value because over time
you become familiar with them and core more readily determine if
things seem about right. For longer time windows you can right-click
on components to view their stats. Presently that is only retained
for up to 24 hours by default but those settings are configurable in
nifi.properties. Those settings control both how often and for how
long those stats are retained. The framework is designed already to
support plugging into a persistent storage for those stats so it is
just a matter of time before a contrib takes care of that. Also, keep
in mind we expect these statistics to be exported to purpose-built
monitoring systems like Ganglia, Ambari, or others such as Riemann
which Ricky Saltzer is working on. There are two ways to export these
stats. First, is the reporting task API which is an extension point
in NiFi. You can push these stats to an external system using that
mechanism. You can also pull the stats via the REST API.
That said, there is work underway for modernizing the look and feel of
NiFi and it is a great time to talk about what would be most helpful
to see in the UI regarding live/historical stats so please feel free
to send drawings/ideas you have on that.
Thanks
Joe
On Fri, Jan 1, 2016 at 11:25 AM, M Singh <ma...@yahoo.com> wrote:
> Hi:
>
> The Nifi processors and components show 5 min status in the UI (In,
> Read/Write, Out, and Tasks/Time) and also in the component stats dialog box.
>
> I wanted to find out if there is a way to
> 1. Customize this time-window,
> 2. Provide options for multiple windows (5 minute, 1 hour, 1 day/etc).
> 3. Compute these dynamically (ie, set the default to 5 minutes, but then
> change a component configuration to see the stats for the previously
> processed events by an hour/etc).
>
> I looked in the admin guide
> (https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html) but
> could not find a property for setting the stats window duration.
>
> If there is an option to set the stats window duration, please let me know.
>
> Thanks
>
> Mans
>
>