You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "Wesley L Lawrence (JIRA)" <ji...@apache.org> on 2017/06/13 15:32:00 UTC
[jira] [Updated] (NIFI-4064) Funnel with no outgoing connections
and a queued Flow File uses high CPU
[ https://issues.apache.org/jira/browse/NIFI-4064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Wesley L Lawrence updated NIFI-4064:
------------------------------------
Description:
I have a simple Flow where a file containing JSON data is read, and converted to Avro.
!FlowWithOneFileInQueue.png!
However, I have the resulting Avro container file parked in a queue going to a Funnel, and it causes my CPU to spike.
!NifiPreEmptyHighCpu.png!
Emptying the queue causes the CPU to return to normal.
!NifiPostEmptyLowCpu.png!
It seems that in the 'ContinuallyRunConnectableTask' class, it does correctly determine that the Funnel shouldn't be run, but since there is both a FlowFile in queue, and a relaiton, it also decides to not yield.
I've added both a fix for this (patch to follow), as well as a new unit test to make sure Funnels without outgoing connections get yielded. With the fix, the CPU spiking is gone.
!WithFixPrePostCpu.png!
was:
I have a simple Flow where a file containing JSON data is read, and converted to Avro.
!FlowWithOneFileInQueue.png|thumbnail!
However, I have the resulting Avro container file parked in a queue going to a Funnel, and it causes my CPU to spike.
!NifiPreEmptyHighCpu.png!
Emptying the queue causes the CPU to return to normal.
!NifiPostEmptyLowCpu.png!
It seems that in the 'ContinuallyRunConnectableTask' class, it does correctly determine that the Funnel shouldn't be run, but since there is both a FlowFile in queue, and a relaiton, it also decides to not yield.
I've added both a fix for this (patch to follow), as well as a new unit test to make sure Funnels without outgoing connections get yielded. With the fix, the CPU spiking is gone.
!WithFixPrePostCpu.png!
> Funnel with no outgoing connections and a queued Flow File uses high CPU
> ------------------------------------------------------------------------
>
> Key: NIFI-4064
> URL: https://issues.apache.org/jira/browse/NIFI-4064
> Project: Apache NiFi
> Issue Type: Bug
> Affects Versions: 1.3.0
> Reporter: Wesley L Lawrence
> Priority: Minor
> Attachments: FlowWithOneFileInQueue.png, NifiPostEmptyLowCpu.png, NifiPreEmptyHighCpu.png, WithFixPrePostCpu.png
>
>
> I have a simple Flow where a file containing JSON data is read, and converted to Avro.
> !FlowWithOneFileInQueue.png!
> However, I have the resulting Avro container file parked in a queue going to a Funnel, and it causes my CPU to spike.
> !NifiPreEmptyHighCpu.png!
> Emptying the queue causes the CPU to return to normal.
> !NifiPostEmptyLowCpu.png!
> It seems that in the 'ContinuallyRunConnectableTask' class, it does correctly determine that the Funnel shouldn't be run, but since there is both a FlowFile in queue, and a relaiton, it also decides to not yield.
> I've added both a fix for this (patch to follow), as well as a new unit test to make sure Funnels without outgoing connections get yielded. With the fix, the CPU spiking is gone.
> !WithFixPrePostCpu.png!
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)