[jira] [Created] (NIFI-7699) MergeContent: Only Route Bins to Failure on Timeout

["Eric Secules (Jira)" <ji...@apache.org>]

Eric Secules created NIFI-7699:
----------------------------------

             Summary: MergeContent: Only Route Bins to Failure on Timeout
                 Key: NIFI-7699
                 URL: https://issues.apache.org/jira/browse/NIFI-7699
             Project: Apache NiFi
          Issue Type: New Feature
          Components: Core Framework
            Reporter: Eric Secules


h1. Use Case

When merging using the defragment strategy it is frustrating when you get a spike in load and then the MergeContent processor starts routing bins to failure before they were able to defragment because the maximum number of bins are already in use and a new flowfile has arrived that doesn't fit into a current bin. 

Rather than evicting the oldest bin, it would be preferable if the new flowfile that doesn't fit into an existing bin were routed to a wait relationship.

Under this feature, bins will be evicted only if they have not completed within a user-specified timeframe.

Under this feature, new flowfiles should not get stuck behind flowfiles that are waiting for a new bin to free up. If this cannot be handled in the processor code, the Usage documentation should identify requirements for prioritizers that are put on input queues.

This feature may also be helpful for the bin-packing strategy. I'll leave that up to the team to decide.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)