You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "Mark Payne (Jira)" <ji...@apache.org> on 2025/05/08 11:55:00 UTC
[jira] [Commented] (NIFI-14366) Allow determining whether or not backpressure is applied to incoming connections
[ https://issues.apache.org/jira/browse/NIFI-14366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17950251#comment-17950251 ]
Mark Payne commented on NIFI-14366:
-----------------------------------
[~vyagh] you can certainly take a stab at this. The NIP ticket is really meant to propose the idea and provide a way to discuss it before formally accepting it as something that the community wants to adopt. The vote happened but I forgot to close the NIP. It is probably a good idea to create a separate Jira for {{nifi-api}} but I would not get overly concerned around the Jira formalities. If you're interested in implementing it, you can add the proposed interface in {{nifi-api}} and then locally change your nifi codebase to look at the SNAPSHOT version of the API.
> Allow determining whether or not backpressure is applied to incoming connections
> --------------------------------------------------------------------------------
>
> Key: NIFI-14366
> URL: https://issues.apache.org/jira/browse/NIFI-14366
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework, NiFi API
> Reporter: Mark Payne
> Priority: Major
>
> There are cases when we need to know whether or not backpressure is applied on incoming connections. For example, in MergeContent, when a Correlation Attribute is used, it is very easy to have a situation in which the Min Number of Entries or Min Bin Size is not met, but in which backpressure is applied. In this case, we have to just wait for the Max Bin Age to elapse in order for MergeContent to progress. This introduces significant artificial delays into our dataflow.
> Instead, it would make sense for MergeContent to determine "all upstream connections have backpressure applied" and as a result immediately merge the "most full bin" in order to make progress.
> In terms of API changes, this would look like:
> {code:java}
> public interface ProcessContext {
> ...
> BackpressureEngagement getBackpressureEngagement();
> }
> public enum BackpressureEngagement {
> // No upstream connections have backpressure engaged
> BACKPRESSURE_NOT_ENGAGED,
> // At least one upstream connection has backpressure engaged but not all
> BACKPRESSURE_PARTIALLY_ENGAGED,
> // All upstream connections have backpressure engaged
> BACKPRESSURE_ENGAGED;
> }{code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)