You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "Bryan Bende (Jira)" <ji...@apache.org> on 2023/06/05 14:36:00 UTC
[jira] [Updated] (NIFI-11464) importing a versioned flow with a nested versioned flow shows nested versioned flow with invalid controller services.
[ https://issues.apache.org/jira/browse/NIFI-11464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bryan Bende updated NIFI-11464:
-------------------------------
Fix Version/s: 2.0.0
1.22.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
> importing a versioned flow with a nested versioned flow shows nested versioned flow with invalid controller services.
> ---------------------------------------------------------------------------------------------------------------------
>
> Key: NIFI-11464
> URL: https://issues.apache.org/jira/browse/NIFI-11464
> Project: Apache NiFi
> Issue Type: Bug
> Components: Flow Versioning
> Affects Versions: 1.21.0
> Environment: nifi 1.21.0 and nifi registry 1.21.0 (on ubuntu 20.04)
> Reporter: Patrick A. Mol
> Assignee: Bryan Bende
> Priority: Major
> Fix For: 2.0.0, 1.22.0
>
> Attachments: exported_flow_versions_pretty.zip, image-2023-04-17-17-34-08-898.png, image-2023-04-17-18-06-16-102.png, nested_versioned_flow_issue.xml, screenprints_reproduction_steps.zip
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> When a flow (reusable_flow_Q) has controller services inherited from the hierarchy (process group reusables) and a version of the flow is stored, the flow version contains the references to these external controller services (as seen in an exported flow version [see below]).
> When this versioned flow is imported in another flow (tenant_flows) the controller services need to be reset to the controller services in the new hierarchy.
> When we have a working flow with the nested versioned flow ready in development we can check this flow into version control.
> When we then deploy the flow in production, the nested versioned flow shows up with invalid components. It shows the external controller service identifiers as stored in the flow version.
> When we then go back to development version of tenant_flows and make a minor change to the nested versioned flow reusable_flow_Q and commit this change to version control.
> Due to this version change, we need to also commit the changes for the tenant_flows process group.
> When we now go back to production, and import this new version of tenant_flows, the nested versioned flow reusable_flow_Q does not have invalid controller services.
> If you have several flows under development using the same reusable components,
> you will likely end up with invalid components after import.
> Depending on the amount of versioned flows used, it could be a lot of work.
> It could also lead to issues when using the ExecuteStateless processor.
> Please see attached template nested_version_flow_issue.xml for a starting point to reproduce the issue. It contains the steps.
> Screenprints are attached in a zip file show the process and diagnosis.
> Controller services identifiers in version 2.
> {code:java}
> $ fgrep -C 4 reusables_avro reusable_flow_Q.json.pretty
> "controllerServices": [
> {
> "identifier": "dc884171-4d75-3854-8604-afab91bd0e60",
> "instanceIdentifier": "8f647d06-0187-1000-4be9-14a61f55d904",
> "name": "reusables_avro_reader",
> "comments": "",
> "type": "org.apache.nifi.avro.AvroReader",
> "bundle": {
> "group": "org.apache.nifi",
> --
> },
> {
> "identifier": "b512b238-cdee-3642-b5cb-0c98d30dd133",
> "instanceIdentifier": "8f64f2c6-0187-1000-7557-ca63c88054dd",
> "name": "reusables_avro_writer",
> "comments": "",
> "type": "org.apache.nifi.avro.AvroRecordSetWriter",
> "bundle": {
> "group": "org.apache.nifi",
> $ head -15 reusable_flow_Q-version-2.json.pretty
> {
> "externalControllerServices": {
> "dc884171-4d75-3854-8604-afab91bd0e60": {
> "identifier": "dc884171-4d75-3854-8604-afab91bd0e60",
> "name": "reusables_avro_reader"
> },
> "b512b238-cdee-3642-b5cb-0c98d30dd133": {
> "identifier": "b512b238-cdee-3642-b5cb-0c98d30dd133",
> "name": "reusables_avro_writer"
> }
> },
> "flowContents": {
> "comments": "used to perform Q ...",
> "componentType": "PROCESS_GROUP",
> "connections": [
> {code}
> Controller services identifiers with version 3 committed in process group "tenant_flows".
> {code:java}
> pmo@hpmo:~/Documents.local/nested_versioned_flows_controller_issue$ fgrep -C 4 tenant_flow_avro tenant_flows-version-1.json.pretty
> ],
> "groupIdentifier": "a984831b-8587-3e17-bbbc-ef4b85c3898d",
> "identifier": "5d9df37d-2a52-3f6e-8cd3-3d3ea9550d22",
> "instanceIdentifier": "8f6cb319-0187-1000-b7fa-83340f7055f7",
> "name": "tenant_flow_avro_writer",
> "properties": {
> "compression-format": "NONE",
> "Schema Write Strategy": "avro-embedded",
> "schema-name": "${schema.name}",
> --
> ],
> "groupIdentifier": "a984831b-8587-3e17-bbbc-ef4b85c3898d",
> "identifier": "8ff96d88-3dc8-30ed-aeb8-757c26a7b807",
> "instanceIdentifier": "8f6c8a94-0187-1000-af54-2fee12838618",
> "name": "tenant_flow_avro_reader",
> "properties": {
> "schema-name": "${schema.name}",
> "cache-size": "1000",
> "schema-access-strategy": "embedded-avro-schema",
> pmo@hpmo:~/Documents.local/nested_versioned_flows_controller_issue$ head -15 reusable_flow_Q-version-3.json.pretty
> {
> "externalControllerServices": {
> "8ff96d88-3dc8-30ed-aeb8-757c26a7b807": {
> "identifier": "8ff96d88-3dc8-30ed-aeb8-757c26a7b807",
> "name": "tenant_flow_avro_reader"
> },
> "5d9df37d-2a52-3f6e-8cd3-3d3ea9550d22": {
> "identifier": "5d9df37d-2a52-3f6e-8cd3-3d3ea9550d22",
> "name": "tenant_flow_avro_writer"
> }
> },
> "flowContents": {
> "comments": "used to perform Q ...",
> "componentType": "PROCESS_GROUP",
> "connections": [
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)