[jira] [Comment Edited] (NIFI-10911) NiFi fails to start due to (likely) corrupted encrypted value(s) in flow.xml.gz

["John Wise (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NIFI-10911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17641432#comment-17641432 ] 

John Wise edited comment on NIFI-10911 at 11/30/22 4:22 PM:
------------------------------------------------------------

[~exceptionfactory] - Unfortunately, due to our customer's secure environment, I can't upload any of the logs or config files, so I'll replicate what I can here.

NiFi version 1.15.3

{{{}nifi.sensitive.props.key=aes/gcm/256  ({}}}also for {{nifi.security.keystorePasswd.protected & nifi.security.truststorePasswd.protected)}}
{{nifi.sensitive.props.algorithm=NIFI_PBKDFS_AES_GCM_256}}
{{nifi.sensitive.props.provider=BC}}

We haven't yet figured out what causes the issue to occur, but it doesn't affect any running NiFi nodes - all of the controller services and processors using encrypted values work as expected, and data is processed without any errors related to those.

I've compared multiple flowfiles in {{/nifi/conf/archive}} & it appears that the "{{{}enc{}}}{}" values get updated every time the flowfile is updated.  I'm not sure if that's the expected behavior, but that's what I'm seeing here.


was (Author: john.wise):
[~exceptionfactory] - Unfortunately, due to our customer's secure environment, I can't upload any of the logs or config files, so I'll replicate what I can here.

NiFi version 1.15.0

{{{}nifi.sensitive.props.key=aes/gcm/256  ({}}}also for {{nifi.security.keystorePasswd.protected & nifi.security.truststorePasswd.protected)}}
{{nifi.sensitive.props.algorithm=NIFI_PBKDFS_AES_GCM_256}}
{{nifi.sensitive.props.provider=BC}}

We haven't yet figured out what causes the issue to occur, but it doesn't affect any running NiFi nodes - all of the controller services and processors using encrypted values work as expected, and data is processed without any errors related to those.

I've compared multiple flowfiles in {{/nifi/conf/archive}} & it appears that the "{{{}enc{}}}{}" values get updated every time the flowfile is updated.  I'm not sure if that's the expected behavior, but that's what I'm seeing here.

> NiFi fails to start due to (likely) corrupted encrypted value(s) in flow.xml.gz
> -------------------------------------------------------------------------------
>
>                 Key: NIFI-10911
>                 URL: https://issues.apache.org/jira/browse/NIFI-10911
>             Project: Apache NiFi
>          Issue Type: Bug
>            Reporter: John Wise
>            Priority: Major
>              Labels: decrypt, failure, startup
>
> Over the past 2-3 weeks, a couple of our clusters have failed to start due to a decryption failure.  nifi-app.log displays "{{{}o.a.n.c.serialization.FlowFromDOMFactory There was a problem decrypting a sensitive flow configuration value.  Check that the nifi.sensitive.props.key value in nifi.properties matches the value used to encrypt the flow.xml.gz file{}}}". 
> In both cases, none of the encryption key values in {{bootstrap.conf}} and {{nifi.properties}} have changed.  The issue appears to be that one, or more, of the "{{{}enc{}}}{}" values in flow.xml.gz have become corrupted.  The issue doesn't present itself until a node is restarted, at which point, NiFi continually fails to start due to the service being configured to auto-restart.
> Ideally, rather than just failing to start, NiFi would still complete the startup & alert the user to any decryption issues, so that they can be fixed.  Also, the log should indicate *which* configuration value(s) it failed to decrypt, to help narrow down where the issue is occurring.
> In the interim, I've been removing the "{{{}enc{}}}{}" values from the flowfile, which allows NiFi to restart & give us the opportunity to manually re-enter the removed values.  It's not ideal, but it does allow us to get our nodes back online.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)