You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "pvillard31 (via GitHub)" <gi...@apache.org> on 2026/05/15 16:59:36 UTC

[PR] NIFI-15945 - Parameter not correctly added to parameter context when having multiple suffixed parameter contexts [nifi]

pvillard31 opened a new pull request, #11255:
URL: https://github.com/apache/nifi/pull/11255

   # Summary
   
   NIFI-15945 - Parameter not correctly added to parameter context when having multiple suffixed parameter contexts
   
   Consider the following scenario: in a registry you have a versioned flow F at version N that represents a versioned process group attached to a parameter context P.
   
   In NiFi, you import that flow three times with the flag that you don't want to re-use the parameter context. You get:
   
   - F_1 with parameter context named "P"
   - F_2 with parameter context named "P (1)"
   - F_3 with parameter context named "P (2)"
   
   Now consider a new version of the flow F, version n+1, where a new parameter Z is added to P.
   
   When you upgrade F_2 or F_3 to the version n+1, the parameter Z is not added to their bounded parameter context.
   
   # Tracking
   
   Please complete the following tracking steps prior to pull request creation.
   
   ### Issue Tracking
   
   - [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created
   
   ### Pull Request Tracking
   
   - Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-00000`
   - Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-00000`
   - Pull request contains [commits signed](https://docs.github.com/en/authentication/managing-commit-signature-verification/signing-commits) with a registered key indicating `Verified` status
   
   ### Pull Request Formatting
   
   - Pull Request based on current revision of the `main` branch
   - Pull Request refers to a feature branch with one commit containing changes
   
   # Verification
   
   Please indicate the verification steps performed prior to pull request creation.
   
   ### Build
   
   - [ ] Build completed using `./mvnw clean install -P contrib-check`
     - [ ] JDK 21
     - [ ] JDK 25
   
   ### Licensing
   
   - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html)
   - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files
   
   ### Documentation
   
   - [ ] Documentation formatting appears as expected in rendered files
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] NIFI-15945 - Parameter not correctly added to parameter context when having multiple suffixed parameter contexts [nifi]

Posted by "pvillard31 (via GitHub)" <gi...@apache.org>.
pvillard31 commented on code in PR #11255:
URL: https://github.com/apache/nifi/pull/11255#discussion_r3253301218


##########
nifi-system-tests/nifi-system-test-suite/src/test/java/org/apache/nifi/tests/system/registry/ParameterContextPreservationIT.java:
##########
@@ -245,6 +245,95 @@ void testNewParameterInInheritedContextSynchronizedDuringUpgrade() throws NiFiCl
         assertTrue(p2NamesAfterUpgrade.contains("paramX"), "paramX should exist on P2 after upgrading to version 2");
     }
 
+    /**
+     * Verifies that a new parameter introduced by a new flow version is applied only to the parameter context
+     * actually bound to the upgraded process group, even when the local flow has multiple deployments of the
+     * same versioned flow with REPLACE-strategy suffix-renamed parameter contexts (P, P (1), P (2)).
+     *
+     * Scenario: Flow F has parameter context P. F is imported three times with the REPLACE strategy, producing
+     * deployments bound to P, P (1), and P (2) respectively. Version 2 of F adds parameter Z to P. Upgrading
+     * the third deployment must apply Z only to P (2); P and P (1) must remain unchanged.
+     */
+    @Test
+    void testNewParameterAppliedOnlyToBoundSuffixedContextDuringUpgrade() throws NiFiClientException, IOException, InterruptedException {
+        final FlowRegistryClientEntity clientEntity = registerClient();
+        final NiFiClientUtil util = getClientUtil();
+
+        // Build the source flow A bound to parameter context P, save as version 1
+        final ParameterContextEntity sourceParamContextP = util.createParameterContext(PARAMETER_CONTEXT_NAME, Map.of(PARAMETER_NAME, PARAMETER_VALUE));
+        final ProcessGroupEntity sourceGroupA = util.createProcessGroup(GROUP_A_NAME, "root");
+        util.setParameterContext(sourceGroupA.getId(), sourceParamContextP);
+
+        final ProcessorEntity processor = util.createProcessor(PROCESSOR_TYPE, sourceGroupA.getId());
+        util.updateProcessorProperties(processor, Collections.singletonMap(PROCESSOR_PROPERTY_TEXT, PARAMETER_REFERENCE));
+        util.setAutoTerminatedRelationships(processor, RELATIONSHIP_SUCCESS);
+
+        final VersionControlInformationEntity vciV1 = util.startVersionControl(sourceGroupA, clientEntity, TEST_FLOWS_BUCKET, FLOW_NAME);
+        final String flowId = vciV1.getVersionControlInformation().getFlowId();
+
+        // Add new parameter paramZ to P and save as version 2
+        final String paramZName = "paramZ";
+        final String paramZValue = "valueZ";
+        final ParameterContextEntity currentSourceP = getNifiClient().getParamContextClient().getParamContext(sourceParamContextP.getId(), false);
+        final ParameterContextUpdateRequestEntity sourceUpdate = util.updateParameterContext(currentSourceP,
+                Map.of(PARAMETER_NAME, PARAMETER_VALUE, paramZName, paramZValue));
+        util.waitForParameterContextRequestToComplete(sourceParamContextP.getId(), sourceUpdate.getRequest().getRequestId());
+
+        final ProcessGroupEntity sourceGroupARefreshed = getNifiClient().getProcessGroupClient().getProcessGroup(sourceGroupA.getId());
+        util.saveFlowVersion(sourceGroupARefreshed, clientEntity, vciV1);
+
+        // Clean up the source flow and parameter context so that subsequent imports start from a clean slate
+        final ProcessGroupEntity sourceForStopVc = getNifiClient().getProcessGroupClient().getProcessGroup(sourceGroupA.getId());
+        getNifiClient().getVersionsClient().stopVersionControl(sourceForStopVc);
+        util.deleteAll(sourceGroupA.getId());
+        final ProcessGroupEntity sourceToDelete = getNifiClient().getProcessGroupClient().getProcessGroup(sourceGroupA.getId());
+        getNifiClient().getProcessGroupClient().deleteProcessGroup(sourceToDelete);
+
+        final ParameterContextEntity sourceContextToDelete = getNifiClient().getParamContextClient().getParamContext(sourceParamContextP.getId(), false);
+        getNifiClient().getParamContextClient().deleteParamContext(sourceParamContextP.getId(),
+                String.valueOf(sourceContextToDelete.getRevision().getVersion()));
+
+        // Import version 1 three times. With the REPLACE strategy, each import creates a new parameter context:
+        // A1 -> P, A2 -> P (1), A3 -> P (2).
+        final ProcessGroupEntity importedA1 = importFlowWithReplaceParameterContext(clientEntity.getId(), flowId, VERSION_1);
+        final String paramContextId1 = getNifiClient().getProcessGroupClient().getProcessGroup(importedA1.getId())
+                .getComponent().getParameterContext().getId();
+
+        final ProcessGroupEntity importedA2 = importFlowWithReplaceParameterContext(clientEntity.getId(), flowId, VERSION_1);
+        final String paramContextId2 = getNifiClient().getProcessGroupClient().getProcessGroup(importedA2.getId())
+                .getComponent().getParameterContext().getId();
+
+        final ProcessGroupEntity importedA3 = importFlowWithReplaceParameterContext(clientEntity.getId(), flowId, VERSION_1);
+        final String paramContextId3 = getNifiClient().getProcessGroupClient().getProcessGroup(importedA3.getId())
+                .getComponent().getParameterContext().getId();
+
+        assertNotEquals(paramContextId1, paramContextId2);
+        assertNotEquals(paramContextId2, paramContextId3);
+        assertNotEquals(paramContextId1, paramContextId3);
+
+        // None of the imported parameter contexts should have paramZ after importing version 1
+        assertFalse(getParameterNames(getNifiClient().getParamContextClient().getParamContext(paramContextId1, false)).contains(paramZName));
+        assertFalse(getParameterNames(getNifiClient().getParamContextClient().getParamContext(paramContextId2, false)).contains(paramZName));
+        assertFalse(getParameterNames(getNifiClient().getParamContextClient().getParamContext(paramContextId3, false)).contains(paramZName));
+
+        // Upgrade only the third deployment (A3, bound to P (2)) to version 2
+        util.changeFlowVersion(importedA3.getId(), VERSION_2);
+
+        // The parameter context bound to A3 must contain the new parameter
+        final ParameterContextEntity context3AfterUpgrade = getNifiClient().getParamContextClient().getParamContext(paramContextId3, false);
+        assertTrue(getParameterNames(context3AfterUpgrade).contains(paramZName),
+                "paramZ should be added to the parameter context bound to the upgraded deployment");
+
+        // The parameter contexts bound to the other deployments must remain unchanged
+        final ParameterContextEntity context1AfterUpgrade = getNifiClient().getParamContextClient().getParamContext(paramContextId1, false);
+        assertFalse(getParameterNames(context1AfterUpgrade).contains(paramZName),
+                "paramZ should not leak into the canonical parameter context bound to a different deployment");

Review Comment:
   done, thanks



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] NIFI-15945 - Parameter not correctly added to parameter context when having multiple suffixed parameter contexts [nifi]

Posted by "exceptionfactory (via GitHub)" <gi...@apache.org>.
exceptionfactory merged PR #11255:
URL: https://github.com/apache/nifi/pull/11255


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] NIFI-15945 - Parameter not correctly added to parameter context when having multiple suffixed parameter contexts [nifi]

Posted by "exceptionfactory (via GitHub)" <gi...@apache.org>.
exceptionfactory commented on code in PR #11255:
URL: https://github.com/apache/nifi/pull/11255#discussion_r3253183155


##########
nifi-system-tests/nifi-system-test-suite/src/test/java/org/apache/nifi/tests/system/registry/ParameterContextPreservationIT.java:
##########
@@ -245,6 +245,95 @@ void testNewParameterInInheritedContextSynchronizedDuringUpgrade() throws NiFiCl
         assertTrue(p2NamesAfterUpgrade.contains("paramX"), "paramX should exist on P2 after upgrading to version 2");
     }
 
+    /**
+     * Verifies that a new parameter introduced by a new flow version is applied only to the parameter context
+     * actually bound to the upgraded process group, even when the local flow has multiple deployments of the
+     * same versioned flow with REPLACE-strategy suffix-renamed parameter contexts (P, P (1), P (2)).
+     *
+     * Scenario: Flow F has parameter context P. F is imported three times with the REPLACE strategy, producing
+     * deployments bound to P, P (1), and P (2) respectively. Version 2 of F adds parameter Z to P. Upgrading
+     * the third deployment must apply Z only to P (2); P and P (1) must remain unchanged.
+     */
+    @Test
+    void testNewParameterAppliedOnlyToBoundSuffixedContextDuringUpgrade() throws NiFiClientException, IOException, InterruptedException {
+        final FlowRegistryClientEntity clientEntity = registerClient();
+        final NiFiClientUtil util = getClientUtil();
+
+        // Build the source flow A bound to parameter context P, save as version 1
+        final ParameterContextEntity sourceParamContextP = util.createParameterContext(PARAMETER_CONTEXT_NAME, Map.of(PARAMETER_NAME, PARAMETER_VALUE));
+        final ProcessGroupEntity sourceGroupA = util.createProcessGroup(GROUP_A_NAME, "root");
+        util.setParameterContext(sourceGroupA.getId(), sourceParamContextP);
+
+        final ProcessorEntity processor = util.createProcessor(PROCESSOR_TYPE, sourceGroupA.getId());
+        util.updateProcessorProperties(processor, Collections.singletonMap(PROCESSOR_PROPERTY_TEXT, PARAMETER_REFERENCE));
+        util.setAutoTerminatedRelationships(processor, RELATIONSHIP_SUCCESS);
+
+        final VersionControlInformationEntity vciV1 = util.startVersionControl(sourceGroupA, clientEntity, TEST_FLOWS_BUCKET, FLOW_NAME);
+        final String flowId = vciV1.getVersionControlInformation().getFlowId();
+
+        // Add new parameter paramZ to P and save as version 2
+        final String paramZName = "paramZ";
+        final String paramZValue = "valueZ";
+        final ParameterContextEntity currentSourceP = getNifiClient().getParamContextClient().getParamContext(sourceParamContextP.getId(), false);
+        final ParameterContextUpdateRequestEntity sourceUpdate = util.updateParameterContext(currentSourceP,
+                Map.of(PARAMETER_NAME, PARAMETER_VALUE, paramZName, paramZValue));
+        util.waitForParameterContextRequestToComplete(sourceParamContextP.getId(), sourceUpdate.getRequest().getRequestId());
+
+        final ProcessGroupEntity sourceGroupARefreshed = getNifiClient().getProcessGroupClient().getProcessGroup(sourceGroupA.getId());
+        util.saveFlowVersion(sourceGroupARefreshed, clientEntity, vciV1);
+
+        // Clean up the source flow and parameter context so that subsequent imports start from a clean slate
+        final ProcessGroupEntity sourceForStopVc = getNifiClient().getProcessGroupClient().getProcessGroup(sourceGroupA.getId());
+        getNifiClient().getVersionsClient().stopVersionControl(sourceForStopVc);
+        util.deleteAll(sourceGroupA.getId());
+        final ProcessGroupEntity sourceToDelete = getNifiClient().getProcessGroupClient().getProcessGroup(sourceGroupA.getId());
+        getNifiClient().getProcessGroupClient().deleteProcessGroup(sourceToDelete);
+
+        final ParameterContextEntity sourceContextToDelete = getNifiClient().getParamContextClient().getParamContext(sourceParamContextP.getId(), false);
+        getNifiClient().getParamContextClient().deleteParamContext(sourceParamContextP.getId(),
+                String.valueOf(sourceContextToDelete.getRevision().getVersion()));
+
+        // Import version 1 three times. With the REPLACE strategy, each import creates a new parameter context:
+        // A1 -> P, A2 -> P (1), A3 -> P (2).
+        final ProcessGroupEntity importedA1 = importFlowWithReplaceParameterContext(clientEntity.getId(), flowId, VERSION_1);
+        final String paramContextId1 = getNifiClient().getProcessGroupClient().getProcessGroup(importedA1.getId())
+                .getComponent().getParameterContext().getId();
+
+        final ProcessGroupEntity importedA2 = importFlowWithReplaceParameterContext(clientEntity.getId(), flowId, VERSION_1);
+        final String paramContextId2 = getNifiClient().getProcessGroupClient().getProcessGroup(importedA2.getId())
+                .getComponent().getParameterContext().getId();
+
+        final ProcessGroupEntity importedA3 = importFlowWithReplaceParameterContext(clientEntity.getId(), flowId, VERSION_1);
+        final String paramContextId3 = getNifiClient().getProcessGroupClient().getProcessGroup(importedA3.getId())
+                .getComponent().getParameterContext().getId();
+
+        assertNotEquals(paramContextId1, paramContextId2);
+        assertNotEquals(paramContextId2, paramContextId3);
+        assertNotEquals(paramContextId1, paramContextId3);
+
+        // None of the imported parameter contexts should have paramZ after importing version 1
+        assertFalse(getParameterNames(getNifiClient().getParamContextClient().getParamContext(paramContextId1, false)).contains(paramZName));
+        assertFalse(getParameterNames(getNifiClient().getParamContextClient().getParamContext(paramContextId2, false)).contains(paramZName));
+        assertFalse(getParameterNames(getNifiClient().getParamContextClient().getParamContext(paramContextId3, false)).contains(paramZName));
+
+        // Upgrade only the third deployment (A3, bound to P (2)) to version 2
+        util.changeFlowVersion(importedA3.getId(), VERSION_2);
+
+        // The parameter context bound to A3 must contain the new parameter
+        final ParameterContextEntity context3AfterUpgrade = getNifiClient().getParamContextClient().getParamContext(paramContextId3, false);
+        assertTrue(getParameterNames(context3AfterUpgrade).contains(paramZName),
+                "paramZ should be added to the parameter context bound to the upgraded deployment");
+
+        // The parameter contexts bound to the other deployments must remain unchanged
+        final ParameterContextEntity context1AfterUpgrade = getNifiClient().getParamContextClient().getParamContext(paramContextId1, false);
+        assertFalse(getParameterNames(context1AfterUpgrade).contains(paramZName),
+                "paramZ should not leak into the canonical parameter context bound to a different deployment");

Review Comment:
   The comments are duplicative of the messages in the assert methods, so I recommend removing the comments.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org