[jira] [Created] (NIFI-16446) Swap in FlowFiles when active queue has room instead of when active queue empty

["Mark Payne (Jira)" <ji...@apache.org>]

Mark Payne created NIFI-16446:
---------------------------------

             Summary: Swap in FlowFiles when active queue has room instead of when active queue empty
                 Key: NIFI-16446
                 URL: https://issues.apache.org/jira/browse/NIFI-16446
             Project: Apache NiFi
          Issue Type: Improvement
          Components: Core Framework
            Reporter: Mark Payne
            Assignee: Mark Payne


Currently, when there are swapped FlowFiles, we only swap them in when the active queue is empty. For processors where we want to grab a (potentially large) batch of FlowFiles from the queue, this limits how many FlowFiles we can pull.
For example, if a Processor grabs all FlowFiles where attribute A = XYZ and there are 50,000 FlowFiles queued, it can only access those in the top 10,000 FlowFiles. The next iteration won't be able to access any because they are not in the active queue, even if there are many ready to be swapped in.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)