[jira] [Commented] (NIFI-4155) Expand EnforceOrder capability to cluster

["Mark Payne (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NIFI-4155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16163568#comment-16163568 ] 

Mark Payne commented on NIFI-4155:
----------------------------------

[~ijokarumawak] can you describe the use case for this change? This was developed originally using only Local state, because we can use back pressure to ensure that only a single FlowFile is transferred. As a result, we can ensure that if we have a flow such as
{code}
EnforceOrder ----- Connection A -----> Processor B
{code}
And we set Connection A to have backpressure of 1 FlowFile, we know that Processor B will process the FlowFiles as intended on that node.

However, if we use a Cluster-wide State Management, we cannot guarantee backpressure of 1 FlowFile in the queue (there will be N flowfiles in the queue, where N is the number of nodes in the cluster) simultaneously. I.e., Processor B will be processing one FlowFile on each node at the same time. I think this defeats the purpose of the EnforceOrder processor. Though I could be missing something...

> Expand EnforceOrder capability to cluster
> -----------------------------------------
>
>                 Key: NIFI-4155
>                 URL: https://issues.apache.org/jira/browse/NIFI-4155
>             Project: Apache NiFi
>          Issue Type: Improvement
>          Components: Extensions
>            Reporter: Koji Kawamura
>            Assignee: Koji Kawamura
>
> Currently, EnforceOrder is able to enforce FlowFile ordering within a NiFi node, and it uses local managed state to track progress.
> If it is configurable which state management to use from Local and Cluster, EnforceOrder would be also able to enforce ordering across cluster with cluster scope state management. It would be useful for some use-cases.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)