You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by James <ge...@gmail.com> on 2020/05/19 07:41:37 UTC
Creating parameters from exported template?
Hi
For the life of me, I can't find anything related to exporting templates to new systems/environments along with the parameters.
I don't need the parameter values necessarily, but, importing a new template and then needing to create the parameters manually while setting them seems a little tedious. Is this only possible using a registry perhaps?
Any guidance would be helpful.
thanks
Re: Creating parameters from exported template?
Posted by Bryan Bende <bb...@gmail.com>.
Hi Nathan,
Right now the git repo behind registry is mostly write-only, meaning if you
add something to it outside of going through the given NiFi Registry
instance, it won't be visible.
The only exception to this is when starting a brand new registry that has
an empty metadata database. So if you ensure the DB is new/wiped every time
that you export the git repo to production, then when you start registry
the first time it will rebuild the database from the git repo and it should
work.
Thanks,
Bryan
On Tue, May 19, 2020 at 8:46 AM <na...@bt.com> wrote:
> Hi Bryan,
>
>
>
> Would the same also work by using two separate git persistence providers
> pointed at two different git repositories? We have a remote production
> instance with no remote access (effectively airgapped).
>
>
>
> Our plan is when we deploy to production, provide a full exported copy of
> the git repository, and it’s history to a locally running instance of a git
> server in production. Which we will then configure the production NiFi
> Registry to use that export of the git repository for read-only operations.
>
>
>
> So two separate Nifi Registries with two independent git repo’s, the
> production one as a cloned export of the Dev/Test Git Repo.
>
>
>
> We are planning on testing this within the next few weeks.
>
>
>
> Kind Regards,
>
>
>
> Nathan
>
>
>
> *From:* Bryan Bende [mailto:bbende@gmail.com]
> *Sent:* 19 May 2020 15:36
> *To:* users@nifi.apache.org
> *Subject:* Re: Creating parameters from exported template?
>
>
>
> John,
>
>
>
> You can have a nifi registry on each network and use Nifi CLI to export
> and import the flows between registries. There is one limitation in that
> nested versioning doesn’t currently work when doing that.
>
>
>
> Also, the parameter contexts do live outside the flow at the controller
> level, however in order to use one of them, the use needs read access to
> the context and write access to the process group. So you can setup fine
> grained policies on each context to restrict who can use it.
>
>
>
> Thanks,
>
>
>
> Bryan
>
>
>
> On Tue, May 19, 2020 at 8:26 AM Bryan Bende <bb...@gmail.com> wrote:
>
> The best option for moving flows across environments would be versioned
> flows in Nifi registry, which have full support for parameters.
>
>
>
> On Tue, May 19, 2020 at 8:01 AM Etienne Jouvin <la...@gmail.com>
> wrote:
>
> Hum, did not see that, and because I am still using "old" (1.9.2) version
> where parameters are not available.
>
>
>
> But I understand like you
>
>
>
> Le mar. 19 mai 2020 à 13:56, James <ge...@gmail.com> a écrit :
>
> Hi
>
> Thanks for the response.
>
> According to this, it's not recommended:
> https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#Variables
>
> "Variables and the nifi.variable.registry.properties property will be
> removed in a future release. As a result, it is highly recommended to
> switch to Parameters."
>
> Or am I thinking of the wrong thing?
>
> thanks
>
> On 2020/05/19 08:11:14, Etienne Jouvin <la...@gmail.com> wrote:
> > Hi.
> >
> > In this case why don't you use variables ?
> >
> > Etienne
> >
> >
> > Le mar. 19 mai 2020 à 09:41, James <ge...@gmail.com> a écrit :
> >
> > > Hi
> > >
> > > For the life of me, I can't find anything related to exporting
> templates
> > > to new systems/environments along with the parameters.
> > >
> > > I don't need the parameter values necessarily, but, importing a new
> > > template and then needing to create the parameters manually while
> setting
> > > them seems a little tedious. Is this only possible using a registry
> perhaps?
> > >
> > > Any guidance would be helpful.
> > >
> > > thanks
> > >
> >
>
> --
>
> Sent from Gmail Mobile
>
> --
>
> Sent from Gmail Mobile
>
RE: Creating parameters from exported template?
Posted by na...@bt.com.
Hi Bryan,
Would the same also work by using two separate git persistence providers pointed at two different git repositories? We have a remote production instance with no remote access (effectively airgapped).
Our plan is when we deploy to production, provide a full exported copy of the git repository, and it’s history to a locally running instance of a git server in production. Which we will then configure the production NiFi Registry to use that export of the git repository for read-only operations.
So two separate Nifi Registries with two independent git repo’s, the production one as a cloned export of the Dev/Test Git Repo.
We are planning on testing this within the next few weeks.
Kind Regards,
Nathan
From: Bryan Bende [mailto:bbende@gmail.com]
Sent: 19 May 2020 15:36
To: users@nifi.apache.org
Subject: Re: Creating parameters from exported template?
John,
You can have a nifi registry on each network and use Nifi CLI to export and import the flows between registries. There is one limitation in that nested versioning doesn’t currently work when doing that.
Also, the parameter contexts do live outside the flow at the controller level, however in order to use one of them, the use needs read access to the context and write access to the process group. So you can setup fine grained policies on each context to restrict who can use it.
Thanks,
Bryan
On Tue, May 19, 2020 at 8:26 AM Bryan Bende <bb...@gmail.com>> wrote:
The best option for moving flows across environments would be versioned flows in Nifi registry, which have full support for parameters.
On Tue, May 19, 2020 at 8:01 AM Etienne Jouvin <la...@gmail.com>> wrote:
Hum, did not see that, and because I am still using "old" (1.9.2) version where parameters are not available.
But I understand like you
Le mar. 19 mai 2020 à 13:56, James <ge...@gmail.com>> a écrit :
Hi
Thanks for the response.
According to this, it's not recommended: https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#Variables
"Variables and the nifi.variable.registry.properties property will be removed in a future release. As a result, it is highly recommended to switch to Parameters."
Or am I thinking of the wrong thing?
thanks
On 2020/05/19 08:11:14, Etienne Jouvin <la...@gmail.com>> wrote:
> Hi.
>
> In this case why don't you use variables ?
>
> Etienne
>
>
> Le mar. 19 mai 2020 à 09:41, James <ge...@gmail.com>> a écrit :
>
> > Hi
> >
> > For the life of me, I can't find anything related to exporting templates
> > to new systems/environments along with the parameters.
> >
> > I don't need the parameter values necessarily, but, importing a new
> > template and then needing to create the parameters manually while setting
> > them seems a little tedious. Is this only possible using a registry perhaps?
> >
> > Any guidance would be helpful.
> >
> > thanks
> >
>
--
Sent from Gmail Mobile
--
Sent from Gmail Mobile
Re: Creating parameters from exported template?
Posted by Bryan Bende <bb...@gmail.com>.
John,
You can have a nifi registry on each network and use Nifi CLI to export and
import the flows between registries. There is one limitation in that nested
versioning doesn’t currently work when doing that.
Also, the parameter contexts do live outside the flow at the controller
level, however in order to use one of them, the use needs read access to
the context and write access to the process group. So you can setup fine
grained policies on each context to restrict who can use it.
Thanks,
Bryan
On Tue, May 19, 2020 at 8:26 AM Bryan Bende <bb...@gmail.com> wrote:
> The best option for moving flows across environments would be versioned
> flows in Nifi registry, which have full support for parameters.
>
> On Tue, May 19, 2020 at 8:01 AM Etienne Jouvin <la...@gmail.com>
> wrote:
>
>> Hum, did not see that, and because I am still using "old" (1.9.2) version
>> where parameters are not available.
>>
>> But I understand like you
>>
>> Le mar. 19 mai 2020 à 13:56, James <ge...@gmail.com> a écrit :
>>
>>> Hi
>>>
>>> Thanks for the response.
>>>
>>> According to this, it's not recommended:
>>> https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#Variables
>>>
>>> "Variables and the nifi.variable.registry.properties property will be
>>> removed in a future release. As a result, it is highly recommended to
>>> switch to Parameters."
>>>
>>> Or am I thinking of the wrong thing?
>>>
>>> thanks
>>>
>>> On 2020/05/19 08:11:14, Etienne Jouvin <la...@gmail.com> wrote:
>>> > Hi.
>>> >
>>> > In this case why don't you use variables ?
>>> >
>>> > Etienne
>>> >
>>> >
>>> > Le mar. 19 mai 2020 à 09:41, James <ge...@gmail.com> a écrit :
>>> >
>>> > > Hi
>>> > >
>>> > > For the life of me, I can't find anything related to exporting
>>> templates
>>> > > to new systems/environments along with the parameters.
>>> > >
>>> > > I don't need the parameter values necessarily, but, importing a new
>>> > > template and then needing to create the parameters manually while
>>> setting
>>> > > them seems a little tedious. Is this only possible using a registry
>>> perhaps?
>>> > >
>>> > > Any guidance would be helpful.
>>> > >
>>> > > thanks
>>> > >
>>> >
>>>
>> --
> Sent from Gmail Mobile
>
--
Sent from Gmail Mobile
Re: Creating parameters from exported template?
Posted by Bryan Bende <bb...@gmail.com>.
The best option for moving flows across environments would be versioned
flows in Nifi registry, which have full support for parameters.
On Tue, May 19, 2020 at 8:01 AM Etienne Jouvin <la...@gmail.com>
wrote:
> Hum, did not see that, and because I am still using "old" (1.9.2) version
> where parameters are not available.
>
> But I understand like you
>
> Le mar. 19 mai 2020 à 13:56, James <ge...@gmail.com> a écrit :
>
>> Hi
>>
>> Thanks for the response.
>>
>> According to this, it's not recommended:
>> https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#Variables
>>
>> "Variables and the nifi.variable.registry.properties property will be
>> removed in a future release. As a result, it is highly recommended to
>> switch to Parameters."
>>
>> Or am I thinking of the wrong thing?
>>
>> thanks
>>
>> On 2020/05/19 08:11:14, Etienne Jouvin <la...@gmail.com> wrote:
>> > Hi.
>> >
>> > In this case why don't you use variables ?
>> >
>> > Etienne
>> >
>> >
>> > Le mar. 19 mai 2020 à 09:41, James <ge...@gmail.com> a écrit :
>> >
>> > > Hi
>> > >
>> > > For the life of me, I can't find anything related to exporting
>> templates
>> > > to new systems/environments along with the parameters.
>> > >
>> > > I don't need the parameter values necessarily, but, importing a new
>> > > template and then needing to create the parameters manually while
>> setting
>> > > them seems a little tedious. Is this only possible using a registry
>> perhaps?
>> > >
>> > > Any guidance would be helpful.
>> > >
>> > > thanks
>> > >
>> >
>>
> --
Sent from Gmail Mobile
Re: Creating parameters from exported template?
Posted by John McGinn <am...@yahoo.com>.
Etienne,
I'm in the same boat as you. Currently using 1.9.2, and haven't upgraded yet, so only know of variables. I'm also concerned about migrating processor groups, as templates, from one environment to another, which cannot be done via NiFi Registry due to different networks/companies/firewalls. Currently, the variables are included as the part of the template, but unsure how the parameter context will be involved in the template. As well, those variables/parameters are now global, and not self-contained in a processor group or template, so any processor group can use any parameter context on a whim, rather than having to create/define it locally. I'm trying to plan out time for investigating the newer versions, and how parameter contexts will need to be used going forward.
John
On Tuesday, May 19, 2020, 08:01:58 AM EDT, Etienne Jouvin <la...@gmail.com> wrote:
Hum, did not see that, and because I am still using "old" (1.9.2) version where parameters are not available.
But I understand like you
Le mar. 19 mai 2020 à 13:56, James <ge...@gmail.com> a écrit :
Hi
Thanks for the response.
According to this, it's not recommended: https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#Variables
"Variables and the nifi.variable.registry.properties property will be removed in a future release. As a result, it is highly recommended to switch to Parameters."
Or am I thinking of the wrong thing?
thanks
On 2020/05/19 08:11:14, Etienne Jouvin <la...@gmail.com> wrote:
> Hi.
>
> In this case why don't you use variables ?
>
> Etienne
>
>
> Le mar. 19 mai 2020 à 09:41, James <ge...@gmail.com> a écrit :
>
> > Hi
> >
> > For the life of me, I can't find anything related to exporting templates
> > to new systems/environments along with the parameters.
> >
> > I don't need the parameter values necessarily, but, importing a new
> > template and then needing to create the parameters manually while setting
> > them seems a little tedious. Is this only possible using a registry perhaps?
> >
> > Any guidance would be helpful.
> >
> > thanks
> >
>
Re: Creating parameters from exported template?
Posted by Etienne Jouvin <la...@gmail.com>.
Hum, did not see that, and because I am still using "old" (1.9.2) version
where parameters are not available.
But I understand like you
Le mar. 19 mai 2020 à 13:56, James <ge...@gmail.com> a écrit :
> Hi
>
> Thanks for the response.
>
> According to this, it's not recommended:
> https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#Variables
>
> "Variables and the nifi.variable.registry.properties property will be
> removed in a future release. As a result, it is highly recommended to
> switch to Parameters."
>
> Or am I thinking of the wrong thing?
>
> thanks
>
> On 2020/05/19 08:11:14, Etienne Jouvin <la...@gmail.com> wrote:
> > Hi.
> >
> > In this case why don't you use variables ?
> >
> > Etienne
> >
> >
> > Le mar. 19 mai 2020 à 09:41, James <ge...@gmail.com> a écrit :
> >
> > > Hi
> > >
> > > For the life of me, I can't find anything related to exporting
> templates
> > > to new systems/environments along with the parameters.
> > >
> > > I don't need the parameter values necessarily, but, importing a new
> > > template and then needing to create the parameters manually while
> setting
> > > them seems a little tedious. Is this only possible using a registry
> perhaps?
> > >
> > > Any guidance would be helpful.
> > >
> > > thanks
> > >
> >
>
Re: Creating parameters from exported template?
Posted by James <ge...@gmail.com>.
Hi
Thanks for the response.
According to this, it's not recommended: https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#Variables
"Variables and the nifi.variable.registry.properties property will be removed in a future release. As a result, it is highly recommended to switch to Parameters."
Or am I thinking of the wrong thing?
thanks
On 2020/05/19 08:11:14, Etienne Jouvin <la...@gmail.com> wrote:
> Hi.
>
> In this case why don't you use variables ?
>
> Etienne
>
>
> Le mar. 19 mai 2020 à 09:41, James <ge...@gmail.com> a écrit :
>
> > Hi
> >
> > For the life of me, I can't find anything related to exporting templates
> > to new systems/environments along with the parameters.
> >
> > I don't need the parameter values necessarily, but, importing a new
> > template and then needing to create the parameters manually while setting
> > them seems a little tedious. Is this only possible using a registry perhaps?
> >
> > Any guidance would be helpful.
> >
> > thanks
> >
>
Re: Creating parameters from exported template?
Posted by Etienne Jouvin <la...@gmail.com>.
Hi.
In this case why don't you use variables ?
Etienne
Le mar. 19 mai 2020 à 09:41, James <ge...@gmail.com> a écrit :
> Hi
>
> For the life of me, I can't find anything related to exporting templates
> to new systems/environments along with the parameters.
>
> I don't need the parameter values necessarily, but, importing a new
> template and then needing to create the parameters manually while setting
> them seems a little tedious. Is this only possible using a registry perhaps?
>
> Any guidance would be helpful.
>
> thanks
>