You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "Brandon Rhys DeVries (Jira)" <ji...@apache.org> on 2020/08/03 13:43:00 UTC
[jira] [Commented] (NIFI-3566) Node fails to pull flow.xml from
NCM, purges content repo
[ https://issues.apache.org/jira/browse/NIFI-3566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17170039#comment-17170039 ]
Brandon Rhys DeVries commented on NIFI-3566:
--------------------------------------------
[~markap14] It was discovered in 0.x, but an analogous potentially problematic line appears to still be present in the main branch [1]. In RocksDBFlowFileRepository [2] this issue is addressed by not dropping data by default, instead requiring a property be set to affirmatively indicate that dropping data is the desired behavior.
[1] [https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/repository/WriteAheadFlowFileRepository.java#L886]
[2][https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/repository/RocksDBFlowFileRepository.java#L1070-L1078]
> Node fails to pull flow.xml from NCM, purges content repo
> ---------------------------------------------------------
>
> Key: NIFI-3566
> URL: https://issues.apache.org/jira/browse/NIFI-3566
> Project: Apache NiFi
> Issue Type: Bug
> Affects Versions: 0.7.1
> Reporter: Brandon Rhys DeVries
> Priority: Minor
>
> We have an instance were a node was removed from a cluster to address a production data flow issue. During this process, changes were made such that it's flow.xml was different from the cluster (different run states). The general procedure we follow in this case is to remove the Node's flow.xml, and let is pull the "correct" / consistent one from the NCM. However, in this case, something prevented the NCM's flow.xml from propagating to the Node. the Node ended up with an empty flow.xml... and then proceeded to purge all of the content repo with the warning "{} maps to unknown FlowFile Queue {}; this record will be discarded"\[1].
> In cases like this, we should see if we can be a bit more friendly. Specifically, in our case, it would have been preferable to shut down rather than delete the content repo. It would seem to me that if an admin intentionally removes the flow.xml, it would not be unreasonable to make it their responsibility to also remove the content repo (and possibly others...). But cases in which a network hiccup can cause 100% data loss on a node seem bad.
> \[1] https://github.com/apache/nifi/blob/rel/nifi-0.7.1/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/repository/WriteAheadFlowFileRepository.java#L717
--
This message was sent by Atlassian Jira
(v8.3.4#803005)