[jira] [Created] (NIFI-14004) NiFi Stateless Engine memory leak when execution gets stuck a single processor

["Bob Paulin (Jira)" <ji...@apache.org>]

Bob Paulin created NIFI-14004:
---------------------------------

             Summary: NiFi Stateless Engine memory leak when execution gets stuck a single processor
                 Key: NIFI-14004
                 URL: https://issues.apache.org/jira/browse/NIFI-14004
             Project: Apache NiFi
          Issue Type: Bug
          Components: NiFi Stateless
            Reporter: Bob Paulin
         Attachments: NiFi-Stateless-OOM.png

I've hit a scenario with a merge processor that cause it to be executed over and over again (See https://issues.apache.org/jira/browse/NIFI-14003)  When this happens the StandardStatelessFlowCurrent class continuously triggers [1] the processor and for processors that extend BinFiles[2],  StatelessProcessSession objects are created and registered with StandardExecutionProgress createSessions list until memory is exhausted !NiFi-Stateless-OOM.png! .



These sessions appear to be saved in-memory to allow for rollback.  Perhaps we need a limit on the size of this list to prevent the OOM?  Could we automatically rollback if this limit is hit?  Perhaps this is something configurable to be passed in from the DataflowTriggerContext object when StandardExecutionProgress is created [4]?
 [1][https://github.com/apache/nifi/blob/6c4ddf86310187e093070446412c4b2314bd324a/nifi-stateless/nifi-stateless-bundle/nifi-stateless-engine/src/main/java/org/apache/nifi/stateless/flow/StandardStatelessFlowCurrent.java#L273|https://github.com/apache/nifi/blob/6c4ddf86310187e093070446412c4b2314bd324a/nifi-stateless/nifi-stateless-bundle/nifi-stateless-engine/src/main/java/org/apache/nifi/stateless/flow/StandardStatelessFlowCurrent.java#L231]

[2][https://github.com/apache/nifi/blob/6c4ddf86310187e093070446412c4b2314bd324a/nifi-extension-bundles/nifi-extension-utils/nifi-bin-manager/src/main/java/org/apache/nifi/processor/util/bin/BinFiles.java#L271]

[3] [https://github.com/apache/nifi/blob/6c4ddf86310187e093070446412c4b2314bd324a/nifi-stateless/nifi-stateless-bundle/nifi-stateless-engine/src/main/java/org/apache/nifi/stateless/engine/StandardExecutionProgress.java#L108]

[4] [https://github.com/apache/nifi/blob/6c4ddf86310187e093070446412c4b2314bd324a/nifi-stateless/nifi-stateless-bundle/nifi-stateless-engine/src/main/java/org/apache/nifi/stateless/flow/StandardStatelessFlow.java#L511]

 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)