You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by jgunvaldson <jg...@cox.net> on 2021/02/02 00:39:18 UTC
Monitoring NiFi Root Level Controller Services
Hi All,
Root level (canvas) Controller Services - We tend to setup several Root level Controller services for developers that are typically DBCPConnectionPools and DistributedMapCacheClientService and maybe a few other. MOST importantly, these controller services cannot be down and cannot be disabled - must remain Enabled at all times.
We have now had a few outages where upon examination a Controller Service has encountered “something” that caused it to be Disabled or Down.
Is there a standard practice we can use to “Monitor” the controller services and ensure we get alerted if any one of them goes into a Disable state?
What do folks generally think is a good monitoring practice?
We are on
HDF Version 3.4.1.1.
Powered by Apache NiFi Version 1.9.0
1.9.0.3.4.1.1-4 built 05/01/2019 02:15:30 UTC
Tagged nifi-1.9.0-RC2
From 7410fa4 on branch UNKNOWN
Re: Monitoring NiFi Root Level Controller Services
Posted by jgunvaldson <jg...@cox.net>.
Hi Daniel,
And yes I am “John” (Joe was correct) - I have some experience with Nipyapi and yes, will probably be using it - I was wondering about a 3rd party (not DataDog) monitoring service (designed) for working with NiFi as recommendations from the group, but certainly - writing our own monitors to use API is probably what we will do
Best Regards
John
> On Feb 1, 2021, at 10:48 PM, Daniel Chaffelson <ch...@gmail.com> wrote:
>
> Hi John, the python client nipyapi already has convenience methods in canvas.py for this, you are looking for a component status of Running.
>
> On Tue, 2 Feb 2021, 01:18 jgunvaldson, <jgunvaldson@cox.net <ma...@cox.net>> wrote:
> Wiring up a rest client to make a rest request to the NIFI API, seems the following three are possible candidates:
>
> GET /controller-services/{id} Gets a controller service
>
> GET /controller-services/{id}/references Gets a controller service
>
> GET /controller-services/{id}/state Gets the state for a controller service
>
> I’ll do some initial tests with Postman to see what is returned in response json payload (and if 200 means up and running, or just a successful API request)
>
> Sounds good
> John
>
>
>
>> On Feb 1, 2021, at 4:46 PM, Joe Witt <joe.witt@gmail.com <ma...@gmail.com>> wrote:
>>
>> John,
>>
>> You're using a vendor distribution of NiFi. You should contact the vendor.
>>
>> You can certainly monitor the state of a controller service via the REST API. They should either be enabled or not enabled.
>>
>> Thanks
>>
>> On Mon, Feb 1, 2021 at 5:39 PM jgunvaldson <jgunvaldson@cox.net <ma...@cox.net>> wrote:
>> Hi All,
>>
>> Root level (canvas) Controller Services - We tend to setup several Root level Controller services for developers that are typically DBCPConnectionPools and DistributedMapCacheClientService and maybe a few other. MOST importantly, these controller services cannot be down and cannot be disabled - must remain Enabled at all times.
>>
>> We have now had a few outages where upon examination a Controller Service has encountered “something” that caused it to be Disabled or Down.
>>
>> Is there a standard practice we can use to “Monitor” the controller services and ensure we get alerted if any one of them goes into a Disable state?
>>
>> What do folks generally think is a good monitoring practice?
>>
>> We are on
>>
>> HDF Version 3.4.1.1.
>>
>> Powered by Apache NiFi Version 1.9.0
>> 1.9.0.3.4.1.1-4 built 05/01/2019 02:15:30 UTC
>> Tagged nifi-1.9.0-RC2
>> From 7410fa4 on branch UNKNOWN
>>
>>
>
Re: Monitoring NiFi Root Level Controller Services
Posted by Daniel Chaffelson <ch...@gmail.com>.
Hi John, the python client nipyapi already has convenience methods in
canvas.py for this, you are looking for a component status of Running.
On Tue, 2 Feb 2021, 01:18 jgunvaldson, <jg...@cox.net> wrote:
> Wiring up a rest client to make a rest request to the NIFI API, seems the
> following three are possible candidates:
>
>
> - GET /controller-services/{id} Gets a
> controller service
> -
> - GET /controller-services/{id}/references Gets a controller
> service
> -
> - GET /controller-services/{id}/state Gets the state
> for a controller service
>
>
> I’ll do some initial tests with Postman to see what is returned in
> response json payload (and if 200 means up and running, or just a
> successful API request)
>
> Sounds good
> John
>
>
>
> On Feb 1, 2021, at 4:46 PM, Joe Witt <jo...@gmail.com> wrote:
>
> John,
>
> You're using a vendor distribution of NiFi. You should contact the vendor.
>
> You can certainly monitor the state of a controller service via the REST
> API. They should either be enabled or not enabled.
>
> Thanks
>
> On Mon, Feb 1, 2021 at 5:39 PM jgunvaldson <jg...@cox.net> wrote:
>
>> Hi All,
>>
>> Root level (canvas) Controller Services - We tend to setup several Root
>> level Controller services for developers that are typically
>> DBCPConnectionPools and DistributedMapCacheClientService and maybe a few
>> other. MOST importantly, these controller services cannot be down and
>> cannot be disabled - must remain Enabled at all times.
>>
>> We have now had a few outages where upon examination a Controller Service
>> has encountered “something” that caused it to be Disabled or Down.
>>
>> Is there a standard practice we can use to “Monitor” the controller
>> services and ensure we get alerted if any one of them goes into a Disable
>> state?
>>
>> What do folks generally think is a good monitoring practice?
>>
>> We are on
>>
>> HDF Version 3.4.1.1.
>>
>> Powered by Apache NiFi Version 1.9.0
>> 1.9.0.3.4.1.1-4 built 05/01/2019 02:15:30 UTC
>> Tagged nifi-1.9.0-RC2
>> From 7410fa4 on branch UNKNOWN
>>
>>
>>
>
Re: Monitoring NiFi Root Level Controller Services
Posted by jgunvaldson <jg...@cox.net>.
Wiring up a rest client to make a rest request to the NIFI API, seems the following three are possible candidates:
GET /controller-services/{id} Gets a controller service
GET /controller-services/{id}/references Gets a controller service
GET /controller-services/{id}/state Gets the state for a controller service
I’ll do some initial tests with Postman to see what is returned in response json payload (and if 200 means up and running, or just a successful API request)
Sounds good
John
> On Feb 1, 2021, at 4:46 PM, Joe Witt <jo...@gmail.com> wrote:
>
> John,
>
> You're using a vendor distribution of NiFi. You should contact the vendor.
>
> You can certainly monitor the state of a controller service via the REST API. They should either be enabled or not enabled.
>
> Thanks
>
> On Mon, Feb 1, 2021 at 5:39 PM jgunvaldson <jgunvaldson@cox.net <ma...@cox.net>> wrote:
> Hi All,
>
> Root level (canvas) Controller Services - We tend to setup several Root level Controller services for developers that are typically DBCPConnectionPools and DistributedMapCacheClientService and maybe a few other. MOST importantly, these controller services cannot be down and cannot be disabled - must remain Enabled at all times.
>
> We have now had a few outages where upon examination a Controller Service has encountered “something” that caused it to be Disabled or Down.
>
> Is there a standard practice we can use to “Monitor” the controller services and ensure we get alerted if any one of them goes into a Disable state?
>
> What do folks generally think is a good monitoring practice?
>
> We are on
>
> HDF Version 3.4.1.1.
>
> Powered by Apache NiFi Version 1.9.0
> 1.9.0.3.4.1.1-4 built 05/01/2019 02:15:30 UTC
> Tagged nifi-1.9.0-RC2
> From 7410fa4 on branch UNKNOWN
>
>
Re: Monitoring NiFi Root Level Controller Services
Posted by Joe Witt <jo...@gmail.com>.
also sorry for referencing you as 'John' - not sure why I just assumed that
:)
On Mon, Feb 1, 2021 at 5:46 PM Joe Witt <jo...@gmail.com> wrote:
> John,
>
> You're using a vendor distribution of NiFi. You should contact the vendor.
>
> You can certainly monitor the state of a controller service via the REST
> API. They should either be enabled or not enabled.
>
> Thanks
>
> On Mon, Feb 1, 2021 at 5:39 PM jgunvaldson <jg...@cox.net> wrote:
>
>> Hi All,
>>
>> Root level (canvas) Controller Services - We tend to setup several Root
>> level Controller services for developers that are typically
>> DBCPConnectionPools and DistributedMapCacheClientService and maybe a few
>> other. MOST importantly, these controller services cannot be down and
>> cannot be disabled - must remain Enabled at all times.
>>
>> We have now had a few outages where upon examination a Controller Service
>> has encountered “something” that caused it to be Disabled or Down.
>>
>> Is there a standard practice we can use to “Monitor” the controller
>> services and ensure we get alerted if any one of them goes into a Disable
>> state?
>>
>> What do folks generally think is a good monitoring practice?
>>
>> We are on
>>
>> HDF Version 3.4.1.1.
>>
>> Powered by Apache NiFi Version 1.9.0
>> 1.9.0.3.4.1.1-4 built 05/01/2019 02:15:30 UTC
>> Tagged nifi-1.9.0-RC2
>> From 7410fa4 on branch UNKNOWN
>>
>>
>>
Re: Monitoring NiFi Root Level Controller Services
Posted by Joe Witt <jo...@gmail.com>.
John,
You're using a vendor distribution of NiFi. You should contact the vendor.
You can certainly monitor the state of a controller service via the REST
API. They should either be enabled or not enabled.
Thanks
On Mon, Feb 1, 2021 at 5:39 PM jgunvaldson <jg...@cox.net> wrote:
> Hi All,
>
> Root level (canvas) Controller Services - We tend to setup several Root
> level Controller services for developers that are typically
> DBCPConnectionPools and DistributedMapCacheClientService and maybe a few
> other. MOST importantly, these controller services cannot be down and
> cannot be disabled - must remain Enabled at all times.
>
> We have now had a few outages where upon examination a Controller Service
> has encountered “something” that caused it to be Disabled or Down.
>
> Is there a standard practice we can use to “Monitor” the controller
> services and ensure we get alerted if any one of them goes into a Disable
> state?
>
> What do folks generally think is a good monitoring practice?
>
> We are on
>
> HDF Version 3.4.1.1.
>
> Powered by Apache NiFi Version 1.9.0
> 1.9.0.3.4.1.1-4 built 05/01/2019 02:15:30 UTC
> Tagged nifi-1.9.0-RC2
> From 7410fa4 on branch UNKNOWN
>
>
>