You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@nifi.apache.org by "Brandon Rhys DeVries (Jira)" <ji...@apache.org> on 2020/04/29 16:08:00 UTC

[jira] [Comment Edited] (NIFI-1675) DistributeLoad: add overflow relationship

    [ https://issues.apache.org/jira/browse/NIFI-1675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17095561#comment-17095561 ] 

Brandon Rhys DeVries edited comment on NIFI-1675 at 4/29/20, 4:07 PM:
----------------------------------------------------------------------

Upon further thought / discussion, I think my comment above is overthinking things.  If you want a bigger buffer before going to overflow, just increase the back-pressure threshold.  No need to make things more complicated.

To be clear, this should still be a new option for DistributeLoad, as opposed to a new processor.


was (Author: devriesb):
Upon further thought / discussion, I think your original PR approach is best.  If you want a bigger buffer before going to overflow, just increase the back-pressure threshold.  No need to make things more complicated.

> DistributeLoad: add overflow relationship
> -----------------------------------------
>
>                 Key: NIFI-1675
>                 URL: https://issues.apache.org/jira/browse/NIFI-1675
>             Project: Apache NiFi
>          Issue Type: Improvement
>            Reporter: Brandon Rhys DeVries
>            Priority: Minor
>
> We've had cases where a user would like to have backpressure on a relationship, and if that threshold is reached then divert new incoming FlowFiles down an alternate (perhaps less computationally intensive) path.  In thinking about how to do this, DistributeLoad seems like a possibility.  We could potentially allow the addition of a (probably single) relationship with a weight of "0", that is never used unless the strategy is "next available", and all other relationships are unavailable.  This would be the "overflow" relationship, providing the desired behavior.  This is somewhat related to the "Interactive Queue Management"[1] that is progressing, but comes from a slightly different direction.  If anyone has any additional thoughts, pros / cons, etc... please feel free to chime in.
> \[1\] https://cwiki.apache.org/confluence/display/NIFI/Interactive+Queue+Management



--
This message was sent by Atlassian Jira
(v8.3.4#803005)