You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "Lucas Ottersbach (Jira)" <ji...@apache.org> on 2020/04/13 06:37:00 UTC
[jira] [Updated] (NIFI-7348) FlowFiles re-entering a Wait-processor
after they've expired expire immediatelly
[ https://issues.apache.org/jira/browse/NIFI-7348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lucas Ottersbach updated NIFI-7348:
-----------------------------------
Description:
We recently noticed a behaviour of the Wait processor that we thought of to be a bug.
As the attribute WAIT_START_TIMESTAMP is only removed once the FlowFile leaves the processor successfully or failing, it affects FlowFiles that expire the EXPIRATION_DURATION and re-enter the processor.
In case the FlowFile enters the same processor again - after expiring beforehand - it is transported to the expired output immediately, without waiting for the EXPIRATION_DURATION again.
Is this desired behaviour?
I'll attach a very simple demonstration. Just let it run a minute or two and look at the FlowFile attribute "counter" afterwards.
There has been a pull-request addressing a similar issue (NIFI-5892), which resulted in the attribute being removed after success and failure. This case just seems to haven't been thought about back then. Or was there a reason to not clear the attribute after expiration? I couldn't find a mention regarding expiration in the issue.
As this should be a very easy fix I would love to contribute, once you confirm this is not intentional.
*Current workaround:*
simply remove the attribute WAIT_START_TIMESTAMP after the FlowFile leaves the Wait processor, e.g. using an UpdateAttribute processor
*Edit 2020-04-13:*
Also this seems to have the side effect of NOT documenting the repeated processing. There is no provenance entry added when re-entering the processor and expiring immediately, leading to the error being harder to trace.
Because of this I reset the priority to "Major", which seems to be the default anyway.
was:
We recently noticed a behaviour of the Wait processor that we thought of to be a bug.
As the attribute WAIT_START_TIMESTAMP is only removed once the FlowFile leaves the processor successfully or failing, it affects FlowFiles that expire the EXPIRATION_DURATION and re-enter the processor.
In case the FlowFile enters the same processor again - after expiring beforehand - it is transported to the expired output immediately, without waiting for the EXPIRATION_DURATION again.
Is this desired behaviour?
I'll attach a very simple demonstration. Just let it run a minute or two and look at the FlowFile attribute "counter" afterwards.
There has been a pull-request addressing a similar issue (NIFI-5892), which resulted in the attribute being removed after success and failure. This case just seems to haven't been thought about back then. Or was there a reason to not clear the attribute after expiration? I couldn't find a mention regarding expiration in the issue.
As this should be a very easy fix I would love to contribute, once you confirm this is not intentional.
*Current workaround:*
simply remove the attribute WAIT_START_TIMESTAMP after the FlowFile leaves the Wait processor, e.g. using an UpdateAttribute processor
Priority: Major (was: Minor)
> FlowFiles re-entering a Wait-processor after they've expired expire immediatelly
> --------------------------------------------------------------------------------
>
> Key: NIFI-7348
> URL: https://issues.apache.org/jira/browse/NIFI-7348
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Affects Versions: 1.11.4
> Environment: Windows 10 / Ubuntu
> Reporter: Lucas Ottersbach
> Assignee: Lucas Ottersbach
> Priority: Major
> Labels: easyfix
> Attachments: Wait_processor_expiration_issue.xml
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> We recently noticed a behaviour of the Wait processor that we thought of to be a bug.
>
> As the attribute WAIT_START_TIMESTAMP is only removed once the FlowFile leaves the processor successfully or failing, it affects FlowFiles that expire the EXPIRATION_DURATION and re-enter the processor.
> In case the FlowFile enters the same processor again - after expiring beforehand - it is transported to the expired output immediately, without waiting for the EXPIRATION_DURATION again.
> Is this desired behaviour?
>
> I'll attach a very simple demonstration. Just let it run a minute or two and look at the FlowFile attribute "counter" afterwards.
>
> There has been a pull-request addressing a similar issue (NIFI-5892), which resulted in the attribute being removed after success and failure. This case just seems to haven't been thought about back then. Or was there a reason to not clear the attribute after expiration? I couldn't find a mention regarding expiration in the issue.
>
> As this should be a very easy fix I would love to contribute, once you confirm this is not intentional.
>
> *Current workaround:*
> simply remove the attribute WAIT_START_TIMESTAMP after the FlowFile leaves the Wait processor, e.g. using an UpdateAttribute processor
>
> *Edit 2020-04-13:*
> Also this seems to have the side effect of NOT documenting the repeated processing. There is no provenance entry added when re-entering the processor and expiring immediately, leading to the error being harder to trace.
> Because of this I reset the priority to "Major", which seems to be the default anyway.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)