way to gracefully terminate a processor on errors

[chandrashekaran kannan <ch...@hotmail.com>]

Hi

   I am new to NIFI and practice it as my hobby.  I would like to know if there is a setting or anything on the anvil , for NIFI to terminate a processor automatically if it encountered run time errors not under the processor’s control , as in memory shortage, errors due to issues in custom code, thread hang-ups or disk space shortages and the like. Most of these result in bulletin messages and result in the flow not progressing beyond this point.

 Typically I have tended to clear-up the queues , and stop the processor manually and at worse had to restart NIFI and remove custom processor’s from the flow. In some cases I had to deal with password changes on database connectivity areas , which is trivial , but still had no way of bailing out gracefully.

Right now I see success and failure as one of the options which I can connect to on each of the processor’s , but it seems there is no way a hung processor or a processor in distress can give up control to failure after some time.

It would be great if there was an signal/event generated with its associated message or stack trace , which could be bound to an exit point for routing it over ,  seeking attention.

I would appreciate any clues on this topic, as I see quite a lot of articles on retrieving bulletin messages and the like, but am yet to come across one which is about this topic.

Once again this is not a production issue I am facing , it is just some curiosity on NIFI flows.

Regards
c.kannan

Re: way to gracefully terminate a processor on errors

[Mohammed Nadeem <na...@gmail.com>]

Hi Chandrashekaran,

You can use the logic which i generally use to stop/start and monitor
failures from either relationship or bullentins. Use Reporting Tasks from
Global Menu - Controller Settings to monitor bulletins, memory use and other
critical information. These reporting tasks needed to be configured with
certain ports which gets triggered whenever there is a active failures
(bullentins, memory use etc). Connect the configured ports to your dataflow
which handles stopping the processors you would want on failures. Stopping
processors can be achieved through NiFi REST APIs.

Further, there is a nice blog from Pierre on configuring reporting tasks.
https://pierrevillard.com/2017/05/13/monitoring-nifi-site2site-reporting-tasks/

Thanks & Regards,
Nadeem



--
Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/