How to pass variables to nifi Processor

[Rajeswari Raghunathan - Contractor <Ra...@8451.com>]

Hi Team,

We are using NIFI in our company to migrate all data from SFTP server to HDFS.
We have different environment like dev, test, prod and  want to accomplish Continuous integration with NIFI. In order to do so, we need to separate all sensitive data like server hostname,username,password from hardcoding in Processor (eg:-GETSFTP).
Can anyone help me by providing best solution for this problem?

Regards,
Rajeswari

RE: How to pass variables to nifi Processor

["Saxena, Shishir" <Sh...@csx.com>]

Hello,
  Another workaround that can be implemented for properties that implement expression language can be to store these properties in a database table. Retrieve them using ExecuteSQL, Then ConvertAvroToJSON and use EvaluateJSONPath to convert JSON to attributes.

This will only work for properties that support Expression language so you are out of luck with Kerberos Principal/keytab/DB Username/password etc.

Thanks.
Shishir

-----Original Message-----
From: Bryan Bende [mailto:bbende@gmail.com] 
Sent: Wednesday, April 06, 2016 9:18 AM
To: dev@nifi.apache.org
Subject: Re: How to pass variables to nifi Processor

Hello,

For properties that support expression language, they can reference properties passed in through the bootstrap.conf file.
So you could define a property in bootstrap.conf like -Dfoo=abc and in a processor property reference that as $[foo}.

There are plans for this be improved in the future through the idea of a variable registry:
https://cwiki.apache.org/confluence/display/NIFI/Variable+Registry

You may also want to take a look at this project that provides a way to deploy templates from one instance to another:
https://github.com/aperepel/nifi-api-deploy

Thanks,

Bryan


On Tue, Apr 5, 2016 at 1:32 PM, Rajeswari Raghunathan - Contractor < Rajeswari.Raghunathan.Contractor@8451.com> wrote:

> Hi Team,
>
> We are using NIFI in our company to migrate all data from SFTP server 
> to HDFS.
> We have different environment like dev, test, prod and  want to 
> accomplish Continuous integration with NIFI. In order to do so, we 
> need to separate all sensitive data like server 
> hostname,username,password from hardcoding in Processor (eg:-GETSFTP).
> Can anyone help me by providing best solution for this problem?
>
> Regards,
> Rajeswari
>



This email transmission and any accompanying attachments may contain CSX privileged and confidential information intended only for the use of the intended addressee. Any dissemination, distribution, copying or action taken in reliance on the contents of this email by anyone other than the intended recipient is strictly prohibited. If you have received this email in error please immediately delete it and notify sender at the above CSX email address. Sender and CSX accept no liability for any damage caused directly or indirectly by receipt of this email.

Re: How to pass variables to nifi Processor

[Bryan Bende <bb...@gmail.com>]

Hello,

For properties that support expression language, they can reference
properties passed in through the bootstrap.conf file.
So you could define a property in bootstrap.conf like -Dfoo=abc and in a
processor property reference that as $[foo}.

There are plans for this be improved in the future through the idea of a
variable registry:
https://cwiki.apache.org/confluence/display/NIFI/Variable+Registry

You may also want to take a look at this project that provides a way to
deploy templates from one instance to another:
https://github.com/aperepel/nifi-api-deploy

Thanks,

Bryan


On Tue, Apr 5, 2016 at 1:32 PM, Rajeswari Raghunathan - Contractor <
Rajeswari.Raghunathan.Contractor@8451.com> wrote:

> Hi Team,
>
> We are using NIFI in our company to migrate all data from SFTP server to
> HDFS.
> We have different environment like dev, test, prod and  want to accomplish
> Continuous integration with NIFI. In order to do so, we need to separate
> all sensitive data like server hostname,username,password from hardcoding
> in Processor (eg:-GETSFTP).
> Can anyone help me by providing best solution for this problem?
>
> Regards,
> Rajeswari
>