[jira] [Created] (NIFI-1329) ControlRate processor should not be penalizing FlowFiles

["Matthew Clarke (JIRA)" <ji...@apache.org>]

Matthew Clarke created NIFI-1329:
------------------------------------

             Summary: ControlRate processor should not be penalizing FlowFiles
                 Key: NIFI-1329
                 URL: https://issues.apache.org/jira/browse/NIFI-1329
             Project: Apache NiFi
          Issue Type: Bug
          Components: Core Framework
    Affects Versions: 0.4.1
            Reporter: Matthew Clarke
            Priority: Minor


The controlRate processor currently penalizes all FlowFiles still on input queue if flow rate has been reached.  Those penalized FlowFiles will not be processed by the ControlRate processor again until that penalization has expired causing an impact on the desired controlled throughput of the processor. 

Example ControlRate processor setup:
Penalty duration left at default 30 sec
Strategy set to FlowFile count
Max rate set at 100 
time duration set to 1 sec

Let say the input queue to this processor has 2000 files dumped in it. The ControlRate processor will read all 2000 files, allow 100 through and penalize the other 1900 fro 30 seconds.  Assuming no other files enter queue at this point, after 30 seconds another 100 will pass and 1800 will be penalized again.  This pattern continues....

You can work around this defect by changing the penalization default from 30 to say 1 sec.

Why is the "In" stat reflecting reading all files on input queue when only 100 files are actually passing through processor?  <-- this leads to confusion for the users.
Why is penalty being applied here?




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)