You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by Jeremy Taylor <jh...@everwatchsolutions.com> on 2022/01/26 15:16:41 UTC

initial NiFi 1.15.3 start-up issue on mongo-uri sensitive property complaint

Greetings,

I am observing a potential return of ticket 7012 in NiFi 1.15.3.

References:

https://issues.apache.org/jira/browse/NIFI-7012

https://stackoverflow.com/questions/64750710/nifi-fails-to-start-after-setting-sensitive-parameter-into-non-sensitive-propert



On initial startup of NiFi 1.15.3 on my project’s flow, NiFi fails to start due to the below stacktrace. This relates to a sensitive parameter context property for a sensitive field in a MongoDBControllerService. As a result, this appears to me as the same error as the referenced defect ticket above. I tried a couple different things to get past this error, including one from the stackoverflow reference above, but that caused unwanted problems.  The most graceful technique I found was to string-replace my prior version of NiFi (1.13.2) with 1.15.3 within the raw flow XML for at least all the mongo processors’ MongoDBControllerServices, This relates to NiFi-packaged mongo processors and my project’s open-source mongo processors. However, in attempting this string-replace, I admit I may have also string-replaced additional items beyond the various MongoDBControllerServices that had 1.13.2 marked to achieve a successful start-up of NiFi and successful execution of all mongo processors at runtime.  While this change did seem to work, I would like to confirm that it is necessary to do this work-around for successful start-up, or am I missing something?

Environment:

Mac OS x86

Java:  OpenJDK version 1.8.0_322

NiFi: 1.15.3

MongoDB (mongodb-driver-legacy) drivers for custom processors and NiFi-packaged MongoDB processers:  4.3.2, but our prior custom processors had used 3.12.5

MongoDB:  4.2.8





2022-01-25 19:34:02,809 WARN [main] org.apache.nifi.web.server.JettyServer Failed to start web server... shutting down.

org.apache.nifi.controller.serialization.FlowSynchronizationException: java.lang.IllegalArgumentException: The property 'mongo-uri' is a sensitive property, so it can only reference Parameters that are sensitive.

       at org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:306)

       at org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1475)

       at org.apache.nifi.persistence.StandardXMLFlowConfigurationDAO.load(StandardXMLFlowConfigurationDAO.java:89)

       at org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:810)

       at org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:539)

       at org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:67)

       at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:1067)

       at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:5



Regards,

--
Jeremy H. Taylor
Software Developer
ACES Incorporated, an EverWatch Company
http://acesinc.net<http://acesinc.net/>
http://everwatchsolutions.com<http://everwatchsolutions.com/>

Re: initial NiFi 1.15.3 start-up issue on mongo-uri sensitive property complaint

Posted by Jeremy Taylor <jh...@everwatchsolutions.com>.
Hi Matt,
Thanks for taking a look.  I pulled up various custom mongoDB processor code. I also pulled up NiFi locally with one of these processors. Our custom mongoDB processors do point to the NiFi-provided MongoDBControllerService for providing the mongo-uri, the username, and the password. Our mongoDB instances do use a password.  The mongo-uri itself is not sensitive for the parameter context we use and neither is the username, but the password we list in the controller service is marked sensitive in the parameter context, which is valid to do.  I recommend making the password for your parameter context that the MongoDBControllerService uses to be sensitive, and for having a NiFi-provided mongoDB processor use a MongoDBControllerService configured that way to make sure you can reproduce.  Please let me know if you still feel you cannot produce.

Our code for these MongoDB custom processors is open-source. If you get curious, the code is here: https://github.com/everwatchsolutions/aces-nifi-processors-bundle

Does this only happen with your own Mongo processors? --> Our flow uses a large percentage of our own MongoDB processors due to needs.  I do feel the issue was happening on both types of processors.  Going back to be able to verify the situation for sure will be a bit too difficult to do. As we are using the same MongoDBControllerService that is standard, I feel that this theoretically should be able to happen on NiFi-provided mongoDB processors as well, but I could be mistaken. However, I recommend you make the password sensitive in your parameter context for when you are configuring your MongoDBControllerService to make sure you can either reproduce or not.

Is the Mongo URI property sensitive in your component(s)? --> No, the mongo-uri itself is not sensitive in our parameter context.  However, as mentioned above, the password is marked sensitive (for the parameter context) in the same MongoDBControllerService that also has the mongo-uri, but the username is not marked sensitive.  

Regards,
-- 
Jeremy H. Taylor


Software Developer

ACES Incorporated, an EverWatch Company

http://acesinc.net <http://acesinc.net/>
http://everwatchsolutions.com <http://everwatchsolutions.com/>

On 1/26/22, 12:51 PM, "Matt Burgess" <ma...@apache.org> wrote:

    Jeremy,

    I can't reproduce this on the latest main branch (closest to 1.15.3).
    What's weird about that error message is that it says 'mongo-uri' is a
    sensitive property, but it is not.  I set up a Parameter Context (PC)
    with a non-sensitive parameter "muri", set the PC on the root Process
    Group (where the Controller Service (CS) is defined), I didn't get any
    errors about sensitivity. Then I restarted NiFi and everything worked
    as expected.

    Just to see what happens I tried it the other way, with "muri" being
    sensitive. I then got the expected error in the UI that the
    sensitivity of the property doesn't match the sensitivity of the
    parameter. I then stopped NiFi, edited the flow.json, and restarted.
    Instead of crashing, the CS was marked invalid with the aforementioned
    message about mismatched sensitivity.

    Does this only happen with your own Mongo processors? Is the Mongo URI
    property sensitive in your component(s)?

    Regards,
    Matt
    CAUTION: This email originated from outside of EverWatch. Do not click links or open attachments unless you recognize the sender and know the content is safe.

Re: initial NiFi 1.15.3 start-up issue on mongo-uri sensitive property complaint

Posted by Matt Burgess <ma...@apache.org>.
Jeremy,

I can't reproduce this on the latest main branch (closest to 1.15.3).
What's weird about that error message is that it says 'mongo-uri' is a
sensitive property, but it is not.  I set up a Parameter Context (PC)
with a non-sensitive parameter "muri", set the PC on the root Process
Group (where the Controller Service (CS) is defined), I didn't get any
errors about sensitivity. Then I restarted NiFi and everything worked
as expected.

Just to see what happens I tried it the other way, with "muri" being
sensitive. I then got the expected error in the UI that the
sensitivity of the property doesn't match the sensitivity of the
parameter. I then stopped NiFi, edited the flow.json, and restarted.
Instead of crashing, the CS was marked invalid with the aforementioned
message about mismatched sensitivity.

Does this only happen with your own Mongo processors? Is the Mongo URI
property sensitive in your component(s)?

Regards,
Matt