[jira] [Commented] (NIFI-2592) Processor still runs @OnScheduled method when it's running on a non-primary node

["Mark Payne (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NIFI-2592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16930752#comment-16930752 ] 

Mark Payne commented on NIFI-2592:
----------------------------------

One gotcha that we'll want to avoid here is the following pattern, which we have in `AbstractListProcessor`, `ListHDFS`, and possibly others:
 # In `onPropertyModified` check if the modified property is one that would necessitate a clearing of the state (for example, the directory being monitored). If so, set a `resetState` flag.
 # In an `@OnScheduled` method, check the `resetState` flag and if it's set, clear the cluster state. Then clear the flag.

This works currently because when the processor is started, the `@OnScheduled` runs on all nodes at the same state. If we changed this to instead only execute when the node becomes Primary Node, what will happen is that the property will be modified, and the flag will be set. Then the processor will be started and run for some time, perhaps days or even months. Then, when the Primary Node changes (due to the node being restarted or whatever), another node will be elected the Primary Node. Then, the `@OnScheduled` method will be invoked, and it will check its flag, which is already set. It will end up clearing state that was amassed over the time that the other node was running. This is essentially the issue that was occurring in NIFI-6677.

To avoid causing this to happen, I would recommend introducing a new concept on the `PropertyDescriptor` such that the developer can indicate `clearStateOnChange(Scope...)` that indicates that if the value of the property is changed, then state (LOCAL, CLUSTER, or BOTH, according to the provided argument) must be cleared. Then, the framework should handle clearing the state as appropriate.

> Processor still runs @OnScheduled method when it's running on a non-primary node
> --------------------------------------------------------------------------------
>
>                 Key: NIFI-2592
>                 URL: https://issues.apache.org/jira/browse/NIFI-2592
>             Project: Apache NiFi
>          Issue Type: Bug
>          Components: Core Framework
>    Affects Versions: 1.0.0
>            Reporter: Jeff Storck
>            Priority: Major
>
> If a processor is scheduled to run on the primary node only, the instances of the processor running on non-primary nodes still have their methods annotated with @OnScheduled invoked. The framework should not invoke the @OnScheduled methods if the processor isn't supposed to run on that node.
> To reproduce this, set up a cluster with the nodes running on the same server. The ListenHTTP can be scheduled to be run on the primary node only, but other instances of the processor on non-primary nodes will still try to bind the port, resulting in exceptions.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)