[jira] [Commented] (NIFI-3414) Implement an EnforceOrder processor

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NIFI-3414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15959703#comment-15959703 ] 

ASF GitHub Bot commented on NIFI-3414:
--------------------------------------

Github user mattyb149 commented on a diff in the pull request:

    https://github.com/apache/nifi/pull/1496#discussion_r110260826
  
    --- Diff: nifi-mock/src/main/java/org/apache/nifi/util/MockProcessSession.java ---
    @@ -753,23 +763,8 @@ public void transfer(final Collection<FlowFile> flowFiles, final Relationship re
                 transfer(flowFiles);
                 return;
             }
    -        if(!processor.getRelationships().contains(relationship)){
    --- End diff --
    
    I couldn't see why/where this section of code needs to be remove, can you elaborate?


> Implement an EnforceOrder processor
> -----------------------------------
>
>                 Key: NIFI-3414
>                 URL: https://issues.apache.org/jira/browse/NIFI-3414
>             Project: Apache NiFi
>          Issue Type: New Feature
>            Reporter: Matt Burgess
>            Assignee: Koji Kawamura
>
> For some flows, it is imperative that the flow files are processed in a certain order.  The PriorityAttributePrioritizer can be used on a connection to ensure that flow files going through that connection are in priority order, but depending on error-handling, branching, and other flow designs, it is possible for flow files to get out-of-order.
> I propose an EnforceOrder processor, which would be single-threaded and have (at a minimum) the following properties:
> 1) Order Attribute: This would be the name of a flow file attribute from which the current value will be retrieved.
> 2) Initial Value: This property specifies an initial value for the order. The processor is stateful, however, so this property is only used when there is no entry in the state map for current value.
> The processor would store the Initial Value into the state map (if no state map entry exists), then for each incoming flow file, it checks the value in the Order Attribute against the current value.  If the attribute value matches the current value, the flow file is transferred to the "success" relationship, and the current value is incremented in the state map. If the attribute value does not match the current value, the session will be rolled back.
> Using this processor, along with a PriorityAttributePrioritizer on the incoming connection, will allow for out-of-order flow files to have a sort of "barrier", thereby guaranteeing that flow files transferred to the "success" relationship are in the specified order.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)