[jira] [Created] (NIFI-8681) Allow users to configure FlowFile Concurrency more specifically on ProcessGroup

["Alexander Stewart (Jira)" <ji...@apache.org>]

Alexander Stewart created NIFI-8681:
---------------------------------------

             Summary: Allow users to configure FlowFile Concurrency more specifically on ProcessGroup
                 Key: NIFI-8681
                 URL: https://issues.apache.org/jira/browse/NIFI-8681
             Project: Apache NiFi
          Issue Type: Improvement
          Components: Core Framework, Core UI, Documentation & Website
            Reporter: Alexander Stewart


With NIFI-7476, users gained the ability to allow a single FlowFile to pass through a ProcessGroup before the next one is allowed through. I propose that we expand this concept to allow for grouping via FlowFile attribute. That is that there would be some hardcoded FlowFile attribute that incoming FlowFile's would be grouped via, and then only a single FlowFile from a grouping is allowed to be processed by the ProcessGroup at a time. This would essentially work as if there were multiple input queues that the ProcessGroup is pulling from - one for each unique FlowFile attribute value - and a FlowFile can only be pulled off the queue if there is no file from that group currently being processed.

It is often the case that one would want to equitably share resources across some grouping, to prevent one grouping from starving out the others. This suggestion would provide an easy solution to such problems as setting the correct grouping would allow for every group to get the same resources without things getting starved out by one group having sent all of its FlowFiles first.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)