[jira] [Comment Edited] (NIFI-619) update MonitorActivity processor to be cluster friendly

["Koji Kawamura (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NIFI-619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15234852#comment-15234852 ] 

Koji Kawamura edited comment on NIFI-619 at 4/11/16 10:57 AM:
--------------------------------------------------------------

 It maybe ideal to NOT alert when there is less load than the configured cluster capacity, for example in a cluster as written in the description, even if one of two nodes doesn't do any processing at all.

However, in an environment that DFM expects every node should receive incoming flow file regularly, each node should report based on its own activity/inactivity.

I think it'd be best to add a property named 'Monitor across cluster' (boolean) to indicate whether this processor should check inactivity cluster-wide or individual node.

When 'Monitor across cluster' is true, every node update its activity using the cluster level StateProvider. Then the processor running on Primary Node checks cluster-wide inactiveness by checking each node's last activity which is stored on the StateProvider.

We should be careful about updating the StateProvider too frequently. It should update StateProvider only when its activeness changes.



was (Author: ijokarumawak):
 It maybe ideal to NOT alert when there is less load than the configured cluster capacity, for example in a cluster as written in the description, even if one of two nodes doesn't do any processing at all.

However, in an environment that DFM expects every node should receive incoming flow file regularly, each node should report based on its own activity/inactivity.

I think it'd be best to add a property named 'Monitor across cluster' (boolean) to indicate whether this processor should check inactivity cluster-wide or individual node. Additionally, when 'Monitor across cluster' is true, then this processor should be scheduled with 'on Primary Node'.

> update MonitorActivity processor to be cluster friendly
> -------------------------------------------------------
>
>                 Key: NIFI-619
>                 URL: https://issues.apache.org/jira/browse/NIFI-619
>             Project: Apache NiFi
>          Issue Type: Improvement
>            Reporter: Brandon DeVries
>            Priority: Minor
>
> This processor should be able to be used to monitor activity across the cluster.  In its current state, alerting is based on activity of a single node, not the entire cluster.
> For example, in a 2 node cluster, if system A is getting data from a given flow and system B is not, system B will alert for lack of activity even though the flow is functioning "normally".
> The ideal behavior would be fore an alert to be generated only if both systems did not see data in the specified time.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)