[jira] [Updated] (NIFI-9689) Do not schedule processors when all FlowFiles are penalized

["Mark Payne (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/NIFI-9689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Mark Payne updated NIFI-9689:
-----------------------------
    Status: Patch Available  (was: Reopened)

> Do not schedule processors when all FlowFiles are penalized
> -----------------------------------------------------------
>
>                 Key: NIFI-9689
>                 URL: https://issues.apache.org/jira/browse/NIFI-9689
>             Project: Apache NiFi
>          Issue Type: Improvement
>          Components: Core Framework
>            Reporter: Mark Payne
>            Assignee: Mark Payne
>            Priority: Major
>             Fix For: 1.16.0
>
>          Time Spent: 40m
>  Remaining Estimate: 0h
>
> When all FlowFiles in a given queue are penalized, the destination processor is still scheduled to run. As a result, the Processor is triggered, no data is available, and it generally returns. However, this means that the UI shows millions of tasks completed over a short period of time, when no work was actually done.
> Further, it means that we constantly schedule the processor to run as fast as it can, which wastes compute resources. Instead, we should detect that the head of the queue is penalized (and therefore all FlowFiles in the queue are penalized), and treat it the same as we would if the queue were empty, which allows the "bored yield duration" to take effect, significantly reducing CPU cycles.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)