You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "Mark Payne (JIRA)" <ji...@apache.org> on 2016/10/07 15:40:20 UTC

[jira] [Created] (NIFI-2875) Provide a Caching FlowFileQueue that will cache a small number of FlowFiles to improve performance

Mark Payne created NIFI-2875:
--------------------------------

             Summary: Provide a Caching FlowFileQueue that will cache a small number of FlowFiles to improve performance
                 Key: NIFI-2875
                 URL: https://issues.apache.org/jira/browse/NIFI-2875
             Project: Apache NiFi
          Issue Type: Improvement
          Components: Core Framework
            Reporter: Mark Payne


Some Processors expect to pull a very large number of FlowFiles at a very fast rate. When this happens, there can be a great deal of contention of the StandardFlowFileQueue. We should create a CachingFlowFileQueue that will wrap another FlowFileQueue. When poll() is called, it will call poll(int) on the underlying FlowFileQueue to poll some number of FlowFiles (10 or 100, perhaps). It can then cache these FlowFiles so that subsequent calls to poll() returns the cached FlowFiles. This avoid a great deal of lock contention on StandardFlowFileQueue because that queue must perform a significant amount of work within its lock.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)