[jira] [Commented] (NIFI-483) Automatically elect a new primary node to allow rolling restarts

["Mark Payne (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NIFI-483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14393245#comment-14393245 ] 

Mark Payne commented on NIFI-483:
---------------------------------

There are a handful of tickets now, I think, that are fairly closely related and similar to this one. We should probably try to organize them a bit and create an overarching ticket that has some "children."

Probably the most important thing to do is to identify the use cases for what we're trying to accomplish. We definitely need to update the Scheduling Strategy / primary node only to make those separate, disparate ideas. We should be able to say CRON driven or Timer driven, and then separately specify whether the processor is run only on the primary node. Would that solve the use case here?

We don't currently auto-elect a new primary node specifically because if the primary node is disconnected, the cluster manager (NCM) has no way to know whether or not the primary is still doing its thing. It could be that the primary node died completely. Or it could be that there's a network problem between primary node and NCM. Or maybe someone accidentally plugged a hole in a firewall. Or any number of things. If we auto-elected a new primary node, then we could end up with 2 different nodes doing a job that is intended to be done only by a single node. So I guess the moral of the story is that we have to be extremely careful about changing something like this. It's probably best to identify exactly what we are trying to accomplish before changing something like this. But I agree, the cluster management can stand to improve.


> Automatically elect a new primary node to allow rolling restarts
> ----------------------------------------------------------------
>
>                 Key: NIFI-483
>                 URL: https://issues.apache.org/jira/browse/NIFI-483
>             Project: Apache NiFi
>          Issue Type: New Feature
>            Reporter: Ricky Saltzer
>
> In a clustered environment, if you restart the primary node's NiFi instance, all of the processors scheduled on "Primary Node" will automatically be re-executed on start up. This is a problem if you require a processor to only execute once per day, for example.  
> To get around this, you must manually promote a different node to primary before restarting it. I think it would be a worthwhile feature to have the primary node automatically elect another node primary if it's being restarted. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)