You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "Brandon Rhys DeVries (Jira)" <ji...@apache.org> on 2020/04/28 18:56:00 UTC
[jira] [Commented] (NIFI-1675) DistributeLoad: add overflow
relationship
[ https://issues.apache.org/jira/browse/NIFI-1675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17094790#comment-17094790 ]
Brandon Rhys DeVries commented on NIFI-1675:
--------------------------------------------
[~johancr] your PR on the other ticket looked good. The one thought I had was about how strict the overflow should be. In other words, if 11 flowfiles come through this processor "at once", and the following relationship has a back-pressure of 10, should it immediately overflow (even if those 10 flowfiles will process through virtually instantaneously)? Or, should there be a delay of some sort? Essentially, the question is whether you want to check if the following relationship is at the limit at the moment you check, or whether you want to check if the downstream flow is saturated and needs to be routed around.
[~jsizzle], do you have any thoughts?
> DistributeLoad: add overflow relationship
> -----------------------------------------
>
> Key: NIFI-1675
> URL: https://issues.apache.org/jira/browse/NIFI-1675
> Project: Apache NiFi
> Issue Type: Improvement
> Reporter: Brandon Rhys DeVries
> Priority: Minor
>
> We've had cases where a user would like to have backpressure on a relationship, and if that threshold is reached then divert new incoming FlowFiles down an alternate (perhaps less computationally intensive) path. In thinking about how to do this, DistributeLoad seems like a possibility. We could potentially allow the addition of a (probably single) relationship with a weight of "0", that is never used unless the strategy is "next available", and all other relationships are unavailable. This would be the "overflow" relationship, providing the desired behavior. This is somewhat related to the "Interactive Queue Management"[1] that is progressing, but comes from a slightly different direction. If anyone has any additional thoughts, pros / cons, etc... please feel free to chime in.
> \[1\] https://cwiki.apache.org/confluence/display/NIFI/Interactive+Queue+Management
--
This message was sent by Atlassian Jira
(v8.3.4#803005)