[jira] [Updated] (NIFI-14715) NiFi Context Modifications are not tracked as Change (NiFi Registry)

["Josef Zahner (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/NIFI-14715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Josef Zahner updated NIFI-14715:
--------------------------------
    Description: 
It seems that the assigned parameter context and it's siblings are always stored together with the versioned config in the same NiFi Registry Flow. At first glance this seems to be fine. However, the problem starts, as soon as you wanna delete a parameter and the context has been assigned to multiple PGs / NiFi Registry Flows: **
 # *Context changes (eg. adding/deleting a parameter) are not tracked as Changes in the GUI under the PG "Version"* (in the older days for variables this was the case!)  but they will be pushed to the NiFi Registry. So, invisible to see what changed on context side between the last commit and now. This leads us to issue 2 and 3 below.
 # *Revert Issue:* Let us say you have two PGs (PG1 and PG2), both are using the same context "global". You start with the initial commit for both PGs as independent flows to the registry. You then create new versions for PG2 and somewhen in between delete a parameter from "global". So the parameter is gone in the GUI as well as in the NiFi Registry for NiFi Registry Flow PG2 and it will remain in the NiFi Registry Flow PG1. You then make changes to PG1 but before you commit, you decide to do a "Revert local Changes" on the Flow PG1. With that, you are importing again the, in the meantime, deleted parameter, which is very confusing and complicated to handle if the context is assigned to a lot of groups. _We have a "global" context, which is assigned and inherited to nearly every PG in our setup. So every flow has it's on copy of the parameter context from the time when it was commited to the NiFi Registry, but the content (parameters) of the context could have been changed in the meantime. So in fact for us it's nearly impossible to really remove a parameter from this "global" context._
 # *Revert Issue with Versioned Sub PGs:* Let's say we have "PG Parent" and "PG Child". Both are using the same context "global" (contains param_1 & param_2), but each PGs has it's on flow in the NiFi Registry. You initially commit both flows. After that, you change anything on "PG Parent" and remove "param_2" from context "global", then commit "PG Parent". Add another processor to "PG Parent" but instead of "Commit local Changes" you do a "Revert local Changes" on "PG Parent". What happens is, that the earlier deleted "param_2" gets imported again, most likely because it was there on the last "PG Child" commit and it seemed to revert not only the "PG Parent" but as well "PG Child".

 

*We see two potential options to overcome the issue (there will be more for sure):*
 * "Simple Fix: NiFi detects parameter modifications as change in the GUI for NiFi Registry versioned flows. So if you change a context, it will be shown in every versioned flow as a modification and you can commit the newest parameter  ontext into the registry. No mess anymore
 * "More Complicated Fix": Store the Context not within each flow separately, but create a reference and store it centrally within the NiFi Registry. So every NiFi Registry Flow using the same context would only have a reference in it's flow. 

 

 

  was:
It seems that the assigned parameter context and it's siblings are always stored together with the versioned config in the same NiFi Registry Flow. At first glance this seems to be fine. However, the problem starts, as soon as you wanna delete a parameter and the context has been assigned to multiple PGs / NiFi Registry Flows: # *Context changes (eg. adding/deleting a parameter) are not tracked as Changes in the GUI under the PG "Version"* (in the older days for variables this was the case!)  but they will be pushed to the NiFi Registry. So, invisible to see what changed on context side between the last commit and now. This leads us to issue 2 and 3 below.
 # *Revert Issue:* Let us say you have two PGs (PG1 and PG2), both are using the same context "global". You start with the initial commit for both PGs as independent flows to the registry. You then create new versions for PG2 and somewhen in between delete a parameter from "global". So the parameter is gone in the GUI as well as in the NiFi Registry for NiFi Registry Flow PG2 and it will remain in the NiFi Registry Flow PG1. You then make changes to PG1 but before you commit, you decide to do a "Revert local Changes" on the Flow PG1. With that, you are importing again the, in the meantime, deleted parameter, which is very confusing and complicated to handle if the context is assigned to a lot of groups. _We have a "global" context, which is assigned and inherited to nearly every PG in our setup. So every flow has it's on copy of the parameter context from the time when it was commited to the NiFi Registry, but the content (parameters) of the context could have been changed in the meantime. So in fact for us it's nearly impossible to really remove a parameter from this "global" context._
 # *Revert Issue with Versioned Sub PGs:* Let's say we have "PG Parent" and "PG Child". Both are using the same context "global" (contains param_1 & param_2), but each PGs has it's on flow in the NiFi Registry. You initially commit both flows. After that, you change anything on "PG Parent" and remove "param_2" from context "global", then commit "PG Parent". Add another processor to "PG Parent" but instead of "Commit local Changes" you do a "Revert local Changes" on "PG Parent". What happens is, that the earlier deleted "param_2" gets imported again, most likely because it was there on the last "PG Child" commit and it seemed to revert not only the "PG Parent" but as well "PG Child".

 

*We see two potential options to overcome the issue (there will be more for sure):*
 * "Simple Fix: NiFi detects parameter modifications as change in the GUI for NiFi Registry versioned flows. So if you change a context, it will be shown in every versioned flow as a modification and you can commit the newest parameter  ontext into the registry. No mess anymore
 * "More Complicated Fix": Store the Context not within each flow separately, but create a reference and store it centrally within the NiFi Registry. So every NiFi Registry Flow using the same context would only have a reference in it's flow. 

 

 


> NiFi Context Modifications are not tracked as Change (NiFi Registry)
> --------------------------------------------------------------------
>
>                 Key: NIFI-14715
>                 URL: https://issues.apache.org/jira/browse/NIFI-14715
>             Project: Apache NiFi
>          Issue Type: Bug
>          Components: Flow Versioning, NiFi Registry
>    Affects Versions: 2.4.0
>            Reporter: Josef Zahner
>            Priority: Major
>
> It seems that the assigned parameter context and it's siblings are always stored together with the versioned config in the same NiFi Registry Flow. At first glance this seems to be fine. However, the problem starts, as soon as you wanna delete a parameter and the context has been assigned to multiple PGs / NiFi Registry Flows: **
>  # *Context changes (eg. adding/deleting a parameter) are not tracked as Changes in the GUI under the PG "Version"* (in the older days for variables this was the case!)  but they will be pushed to the NiFi Registry. So, invisible to see what changed on context side between the last commit and now. This leads us to issue 2 and 3 below.
>  # *Revert Issue:* Let us say you have two PGs (PG1 and PG2), both are using the same context "global". You start with the initial commit for both PGs as independent flows to the registry. You then create new versions for PG2 and somewhen in between delete a parameter from "global". So the parameter is gone in the GUI as well as in the NiFi Registry for NiFi Registry Flow PG2 and it will remain in the NiFi Registry Flow PG1. You then make changes to PG1 but before you commit, you decide to do a "Revert local Changes" on the Flow PG1. With that, you are importing again the, in the meantime, deleted parameter, which is very confusing and complicated to handle if the context is assigned to a lot of groups. _We have a "global" context, which is assigned and inherited to nearly every PG in our setup. So every flow has it's on copy of the parameter context from the time when it was commited to the NiFi Registry, but the content (parameters) of the context could have been changed in the meantime. So in fact for us it's nearly impossible to really remove a parameter from this "global" context._
>  # *Revert Issue with Versioned Sub PGs:* Let's say we have "PG Parent" and "PG Child". Both are using the same context "global" (contains param_1 & param_2), but each PGs has it's on flow in the NiFi Registry. You initially commit both flows. After that, you change anything on "PG Parent" and remove "param_2" from context "global", then commit "PG Parent". Add another processor to "PG Parent" but instead of "Commit local Changes" you do a "Revert local Changes" on "PG Parent". What happens is, that the earlier deleted "param_2" gets imported again, most likely because it was there on the last "PG Child" commit and it seemed to revert not only the "PG Parent" but as well "PG Child".
>  
> *We see two potential options to overcome the issue (there will be more for sure):*
>  * "Simple Fix: NiFi detects parameter modifications as change in the GUI for NiFi Registry versioned flows. So if you change a context, it will be shown in every versioned flow as a modification and you can commit the newest parameter  ontext into the registry. No mess anymore
>  * "More Complicated Fix": Store the Context not within each flow separately, but create a reference and store it centrally within the NiFi Registry. So every NiFi Registry Flow using the same context would only have a reference in it's flow. 
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)