You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "Mark Payne (JIRA)" <ji...@apache.org> on 2016/10/07 15:40:20 UTC
[jira] [Created] (NIFI-2875) Provide a Caching FlowFileQueue that
will cache a small number of FlowFiles to improve performance
Mark Payne created NIFI-2875:
--------------------------------
Summary: Provide a Caching FlowFileQueue that will cache a small number of FlowFiles to improve performance
Key: NIFI-2875
URL: https://issues.apache.org/jira/browse/NIFI-2875
Project: Apache NiFi
Issue Type: Improvement
Components: Core Framework
Reporter: Mark Payne
Some Processors expect to pull a very large number of FlowFiles at a very fast rate. When this happens, there can be a great deal of contention of the StandardFlowFileQueue. We should create a CachingFlowFileQueue that will wrap another FlowFileQueue. When poll() is called, it will call poll(int) on the underlying FlowFileQueue to poll some number of FlowFiles (10 or 100, perhaps). It can then cache these FlowFiles so that subsequent calls to poll() returns the cached FlowFiles. This avoid a great deal of lock contention on StandardFlowFileQueue because that queue must perform a significant amount of work within its lock.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)