You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nifi.apache.org by Chad Zobrisky <cz...@gmail.com> on 2020/10/19 15:43:57 UTC
Parameter Contexts Expression Language scopes
Hello,
I was configuring an SSL Context Controller Service today and had the
keystores and passwords passed into the container via environment
variables. I thought it would be nice to be able to reference these from
the parameter context. Maybe either giving Parameter Context values the
VARIABLE_REGISTRY scope in the Expression Language, or a new scope for
references external to nifi?
I think for refreshing the Parameter Context on those external changes, it
would require an edit/re-apply just as it does now, and would have to make
sure it is well documented.
I'd be interested in creating a PR for this if the idea makes sense and is
acceptable.
Thanks,
Chad
Re: Parameter Contexts Expression Language scopes
Posted by "Uwe@Moosheimer.com" <Uw...@moosheimer.com>.
Then I must have misunderstood it.
Thanks Bryan for clarification.
However, the idea of Chad makes sense for me.
Mit freundlichen Grüßen / best regards
Kay-Uwe Moosheimer
> Am 19.10.2020 um 20:35 schrieb Bryan Bende <bb...@gmail.com>:
>
> Access to environment variables directly from expression language is
> not being removed.
>
> The discussion is about whether a parameter value should be able to
> use expression language to reference an environment variable.
>
> For example, processor property has #{keystore.password} -> parameter
> "keystore.password" has value "${KEYSTORE_PASSWORD}" which then gets
> the password from an environment variable.
>
>
>
>> On Mon, Oct 19, 2020 at 2:14 PM Uwe@Moosheimer.com <Uw...@moosheimer.com> wrote:
>>
>> Chad,
>>
>> So far I thought that only the NiFi variables are deprecated and access to environment variables will still be possible.
>>
>> If this is not the case, then I agree with you. It should definitely be possible to access environment variables. Otherwise I can't imagine how to refer to the hostname or the current JAVA path or ... or ... or on each node?!
>>
>> Mit freundlichen Grüßen / best regards
>> Kay-Uwe Moosheimer
>>
>>>> Am 19.10.2020 um 20:00 schrieb Chad Zobrisky <cz...@gmail.com>:
>>>
>>> Andy,
>>>
>>> Thanks for the response!
>>>
>>> When I was thinking through this the deprecation of variables was
>>> definitely on my mind but the fact that it already had direct access to
>>> environment variables was the simplest path. I think it does make more
>>> sense to add access to environment variables to the parameter context, or
>>> allowing a specific scope just for environment variables in the
>>> expression language.
>>>
>>> I think giving access to environment variables actually allows more
>>> portability between environments, eg dev, test, prod. Defining those once
>>> and allowing for nifi to pull them in makes sense to me and I think is
>>> common in container environments.
>>>
>>> Looking forward to discussing more and better approaches.
>>> Chad
>>>
>>>> On Mon, Oct 19, 2020 at 1:46 PM Andy LoPresto <al...@apache.org> wrote:
>>>>
>>>> Hi Chad,
>>>>
>>>> Parameters were introduced as a way to deprecate (NiFi) variables
>>>> entirely. I’m not sure that introducing a dependency between the two is a
>>>> positive step forward. I think there is a separate conversation to be had
>>>> about allowing parameters access to environment variables, but I think this
>>>> could introduce problems as parameters are designed for flexibility and
>>>> portability, and moving from a system where a parameter was actually a
>>>> pass-through to an environment variable would cause unexpected problems on
>>>> the destination system.
>>>>
>>>> I think the pros and cons of this need to be clearly enumerated and
>>>> discussed here. Thanks for bringing this up.
>>>>
>>>>
>>>> Andy LoPresto
>>>> alopresto@apache.org
>>>> alopresto.apache@gmail.com
>>>> He/Him
>>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
>>>>
>>>>>> On Oct 19, 2020, at 9:43 AM, Chad Zobrisky <cz...@gmail.com> wrote:
>>>>>
>>>>> Hello,
>>>>>
>>>>> I was configuring an SSL Context Controller Service today and had the
>>>>> keystores and passwords passed into the container via environment
>>>>> variables. I thought it would be nice to be able to reference these from
>>>>> the parameter context. Maybe either giving Parameter Context values the
>>>>> VARIABLE_REGISTRY scope in the Expression Language, or a new scope for
>>>>> references external to nifi?
>>>>>
>>>>> I think for refreshing the Parameter Context on those external changes,
>>>> it
>>>>> would require an edit/re-apply just as it does now, and would have to
>>>> make
>>>>> sure it is well documented.
>>>>>
>>>>> I'd be interested in creating a PR for this if the idea makes sense and
>>>> is
>>>>> acceptable.
>>>>>
>>>>> Thanks,
>>>>> Chad
>>>>
>>>>
>>
Re: Parameter Contexts Expression Language scopes
Posted by Phillip Grenier <ph...@nifi.rocks>.
I also see this as a good idea, if only for transportability. If the dev
environment is in a docker container and wants to use an environment
variable to set the context, but in production where the sensitive values
need more protection they can be set manually or in another way.
Chris Sampson's idea of notifying the users if a value changes is a good
one, and likely enough to handle the "inconsistency" factor if someone
changes the environment variable.
Phillip
On Mon, Oct 19, 2020 at 5:27 PM Uwe@Moosheimer.com <Uw...@moosheimer.com>
wrote:
> If parameters make it possible to use environment variables, then the
> system can be configured in one place. You would not separate between
> expression language and parameters.
>
> It is also possible that an environment variable should be replaced by a
> parameter or vice versa.
> It makes sense to configure this in a central place.
>
> Therefore I think this is a good suggestion.
>
> Mit freundlichen Grüßen / best regards
> Kay-Uwe Moosheimer
>
> > Am 19.10.2020 um 23:18 schrieb Bryan Bende <bb...@gmail.com>:
> >
> > I think there is some confusion based on terminology...
> >
> > A given property has an expression language scope defined, which can
> > be "flow file attributes" or "variable registry only". This was
> > created mostly for documentation purposes so that users could look at
> > a property in the docs and see "expression language supported: true"
> > and know which values they could reference. What it really comes down
> > to in the code is the difference between
> > "someProperty.evaluateAttributeExpressions()" and
> > "someProperty.evaluateAttributeExpressions(flowFile)"... meaning can I
> > reference a value from a flow file or not.
> >
> > In the case of "variable registry only", the value can come from any
> > of the following...
> > a) system properties
> > b) expression language
> > c) process-group variable registry
> > d) file-based variable registry
> >
> > When people have stated that "variables are being deprecated in favor
> > of parameters", they are referring to the last two items (c & d).
> >
> > The reason being that parameters solved several short-comings of those
> > two options...
> >
> > - the ability to store sensitive values encrypted
> > - the ability to reference them from any property using a new syntax
> > #{...}, not long dependent on component developer saying
> > "expressionLanguageSupported(true)" on the descriptor
> > - the ability to create policies for which users/groups could
> > reference a set of parameters
> > - the hierarchical ambiguity of what "${foo}" actually resolves to
> >
> > If you just want to use environment variables, why use parameter
> > contexts? Expression language has always offered access to env vars
> > and still will going forward.
> >
> > The one argument I can see is that not all properties support
> > expression language, so using parameters gives you a way around that.
> >
> >
> >
> >
> >
> >> On Mon, Oct 19, 2020 at 3:58 PM Chris Sampson
> >> <ch...@naimuri.com.invalid> wrote:
> >>
> >> Being based primarily in Docker containers and having experience with
> both
> >> Kubernetes (where secrets such as KESTORE_PASSWORD can be injected as
> >> environment variables or files) and Docker Swarm (which only handles
> >> secrets as environment variables), I'd have definitely been wanting this
> >> before moving from Variables to Parameters if I was still in Swarm (or
> >> Docker Compose/straight up Docker).
> >>
> >> It's certainly possible to script creating/updating Parameters via the
> >> Toolkit/NiPyAPI, but in Docker Swarm that's not so easy (whereas it's
> >> possible as a Job in Kubernetes, for example). So environment variables
> >> could save the day in that instance.
> >>
> >> I guess one likely problem (but no different to how I guess the Variable
> >> Registry uses env vars) would be how NiFi will handle changes to the env
> >> vars - does it:
> >>
> >> - ignore them until instance restart, which could lead to maintainer
> >> confusion (I've changed KEYSTORE_PASSWORD in the env but things are
> still
> >> failing in NiFi)
> >> - alert the maintainer to the fact that the env var has changed and a
> >> Parameter needs updating, with the new value being used after all
> >> associated processors/controllers have been restarted
> >> - automatically attempt to update the parameters by restarting all
> >> associated processors/controllers, which I'd guess would be a bit
> dangerous
> >> for interrupting in-flow data, etc.
> >>
> >>
> >> ---
> >> *Chris Sampson*
> >> IT Consultant
> >> chris.sampson@naimuri.com
> >> <https://www.naimuri.com/>
> >>
> >>
> >>> On Mon, 19 Oct 2020 at 19:35, Bryan Bende <bb...@gmail.com> wrote:
> >>>
> >>> Access to environment variables directly from expression language is
> >>> not being removed.
> >>>
> >>> The discussion is about whether a parameter value should be able to
> >>> use expression language to reference an environment variable.
> >>>
> >>> For example, processor property has #{keystore.password} -> parameter
> >>> "keystore.password" has value "${KEYSTORE_PASSWORD}" which then gets
> >>> the password from an environment variable.
> >>>
> >>>
> >>>
> >>> On Mon, Oct 19, 2020 at 2:14 PM Uwe@Moosheimer.com <Uwe@moosheimer.com
> >
> >>> wrote:
> >>>>
> >>>> Chad,
> >>>>
> >>>> So far I thought that only the NiFi variables are deprecated and
> access
> >>> to environment variables will still be possible.
> >>>>
> >>>> If this is not the case, then I agree with you. It should definitely
> be
> >>> possible to access environment variables. Otherwise I can't imagine
> how to
> >>> refer to the hostname or the current JAVA path or ... or ... or on each
> >>> node?!
> >>>>
> >>>> Mit freundlichen Grüßen / best regards
> >>>> Kay-Uwe Moosheimer
> >>>>
> >>>>> Am 19.10.2020 um 20:00 schrieb Chad Zobrisky <cz...@gmail.com>:
> >>>>>
> >>>>> Andy,
> >>>>>
> >>>>> Thanks for the response!
> >>>>>
> >>>>> When I was thinking through this the deprecation of variables was
> >>>>> definitely on my mind but the fact that it already had direct access
> to
> >>>>> environment variables was the simplest path. I think it does make
> more
> >>>>> sense to add access to environment variables to the parameter
> context,
> >>> or
> >>>>> allowing a specific scope just for environment variables in the
> >>>>> expression language.
> >>>>>
> >>>>> I think giving access to environment variables actually allows more
> >>>>> portability between environments, eg dev, test, prod. Defining those
> >>> once
> >>>>> and allowing for nifi to pull them in makes sense to me and I think
> is
> >>>>> common in container environments.
> >>>>>
> >>>>> Looking forward to discussing more and better approaches.
> >>>>> Chad
> >>>>>
> >>>>>> On Mon, Oct 19, 2020 at 1:46 PM Andy LoPresto <alopresto@apache.org
> >
> >>> wrote:
> >>>>>>
> >>>>>> Hi Chad,
> >>>>>>
> >>>>>> Parameters were introduced as a way to deprecate (NiFi) variables
> >>>>>> entirely. I’m not sure that introducing a dependency between the two
> >>> is a
> >>>>>> positive step forward. I think there is a separate conversation to
> be
> >>> had
> >>>>>> about allowing parameters access to environment variables, but I
> >>> think this
> >>>>>> could introduce problems as parameters are designed for flexibility
> >>> and
> >>>>>> portability, and moving from a system where a parameter was
> actually a
> >>>>>> pass-through to an environment variable would cause unexpected
> >>> problems on
> >>>>>> the destination system.
> >>>>>>
> >>>>>> I think the pros and cons of this need to be clearly enumerated and
> >>>>>> discussed here. Thanks for bringing this up.
> >>>>>>
> >>>>>>
> >>>>>> Andy LoPresto
> >>>>>> alopresto@apache.org
> >>>>>> alopresto.apache@gmail.com
> >>>>>> He/Him
> >>>>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
> >>>>>>
> >>>>>>>> On Oct 19, 2020, at 9:43 AM, Chad Zobrisky <cz...@gmail.com>
> >>> wrote:
> >>>>>>>
> >>>>>>> Hello,
> >>>>>>>
> >>>>>>> I was configuring an SSL Context Controller Service today and had
> the
> >>>>>>> keystores and passwords passed into the container via environment
> >>>>>>> variables. I thought it would be nice to be able to reference these
> >>> from
> >>>>>>> the parameter context. Maybe either giving Parameter Context values
> >>> the
> >>>>>>> VARIABLE_REGISTRY scope in the Expression Language, or a new scope
> >>> for
> >>>>>>> references external to nifi?
> >>>>>>>
> >>>>>>> I think for refreshing the Parameter Context on those external
> >>> changes,
> >>>>>> it
> >>>>>>> would require an edit/re-apply just as it does now, and would have
> to
> >>>>>> make
> >>>>>>> sure it is well documented.
> >>>>>>>
> >>>>>>> I'd be interested in creating a PR for this if the idea makes sense
> >>> and
> >>>>>> is
> >>>>>>> acceptable.
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>> Chad
> >>>>>>
> >>>>>>
> >>>>
> >>>
>
>
Re: Parameter Contexts Expression Language scopes
Posted by "Uwe@Moosheimer.com" <Uw...@moosheimer.com>.
If parameters make it possible to use environment variables, then the system can be configured in one place. You would not separate between expression language and parameters.
It is also possible that an environment variable should be replaced by a parameter or vice versa.
It makes sense to configure this in a central place.
Therefore I think this is a good suggestion.
Mit freundlichen Grüßen / best regards
Kay-Uwe Moosheimer
> Am 19.10.2020 um 23:18 schrieb Bryan Bende <bb...@gmail.com>:
>
> I think there is some confusion based on terminology...
>
> A given property has an expression language scope defined, which can
> be "flow file attributes" or "variable registry only". This was
> created mostly for documentation purposes so that users could look at
> a property in the docs and see "expression language supported: true"
> and know which values they could reference. What it really comes down
> to in the code is the difference between
> "someProperty.evaluateAttributeExpressions()" and
> "someProperty.evaluateAttributeExpressions(flowFile)"... meaning can I
> reference a value from a flow file or not.
>
> In the case of "variable registry only", the value can come from any
> of the following...
> a) system properties
> b) expression language
> c) process-group variable registry
> d) file-based variable registry
>
> When people have stated that "variables are being deprecated in favor
> of parameters", they are referring to the last two items (c & d).
>
> The reason being that parameters solved several short-comings of those
> two options...
>
> - the ability to store sensitive values encrypted
> - the ability to reference them from any property using a new syntax
> #{...}, not long dependent on component developer saying
> "expressionLanguageSupported(true)" on the descriptor
> - the ability to create policies for which users/groups could
> reference a set of parameters
> - the hierarchical ambiguity of what "${foo}" actually resolves to
>
> If you just want to use environment variables, why use parameter
> contexts? Expression language has always offered access to env vars
> and still will going forward.
>
> The one argument I can see is that not all properties support
> expression language, so using parameters gives you a way around that.
>
>
>
>
>
>> On Mon, Oct 19, 2020 at 3:58 PM Chris Sampson
>> <ch...@naimuri.com.invalid> wrote:
>>
>> Being based primarily in Docker containers and having experience with both
>> Kubernetes (where secrets such as KESTORE_PASSWORD can be injected as
>> environment variables or files) and Docker Swarm (which only handles
>> secrets as environment variables), I'd have definitely been wanting this
>> before moving from Variables to Parameters if I was still in Swarm (or
>> Docker Compose/straight up Docker).
>>
>> It's certainly possible to script creating/updating Parameters via the
>> Toolkit/NiPyAPI, but in Docker Swarm that's not so easy (whereas it's
>> possible as a Job in Kubernetes, for example). So environment variables
>> could save the day in that instance.
>>
>> I guess one likely problem (but no different to how I guess the Variable
>> Registry uses env vars) would be how NiFi will handle changes to the env
>> vars - does it:
>>
>> - ignore them until instance restart, which could lead to maintainer
>> confusion (I've changed KEYSTORE_PASSWORD in the env but things are still
>> failing in NiFi)
>> - alert the maintainer to the fact that the env var has changed and a
>> Parameter needs updating, with the new value being used after all
>> associated processors/controllers have been restarted
>> - automatically attempt to update the parameters by restarting all
>> associated processors/controllers, which I'd guess would be a bit dangerous
>> for interrupting in-flow data, etc.
>>
>>
>> ---
>> *Chris Sampson*
>> IT Consultant
>> chris.sampson@naimuri.com
>> <https://www.naimuri.com/>
>>
>>
>>> On Mon, 19 Oct 2020 at 19:35, Bryan Bende <bb...@gmail.com> wrote:
>>>
>>> Access to environment variables directly from expression language is
>>> not being removed.
>>>
>>> The discussion is about whether a parameter value should be able to
>>> use expression language to reference an environment variable.
>>>
>>> For example, processor property has #{keystore.password} -> parameter
>>> "keystore.password" has value "${KEYSTORE_PASSWORD}" which then gets
>>> the password from an environment variable.
>>>
>>>
>>>
>>> On Mon, Oct 19, 2020 at 2:14 PM Uwe@Moosheimer.com <Uw...@moosheimer.com>
>>> wrote:
>>>>
>>>> Chad,
>>>>
>>>> So far I thought that only the NiFi variables are deprecated and access
>>> to environment variables will still be possible.
>>>>
>>>> If this is not the case, then I agree with you. It should definitely be
>>> possible to access environment variables. Otherwise I can't imagine how to
>>> refer to the hostname or the current JAVA path or ... or ... or on each
>>> node?!
>>>>
>>>> Mit freundlichen Grüßen / best regards
>>>> Kay-Uwe Moosheimer
>>>>
>>>>> Am 19.10.2020 um 20:00 schrieb Chad Zobrisky <cz...@gmail.com>:
>>>>>
>>>>> Andy,
>>>>>
>>>>> Thanks for the response!
>>>>>
>>>>> When I was thinking through this the deprecation of variables was
>>>>> definitely on my mind but the fact that it already had direct access to
>>>>> environment variables was the simplest path. I think it does make more
>>>>> sense to add access to environment variables to the parameter context,
>>> or
>>>>> allowing a specific scope just for environment variables in the
>>>>> expression language.
>>>>>
>>>>> I think giving access to environment variables actually allows more
>>>>> portability between environments, eg dev, test, prod. Defining those
>>> once
>>>>> and allowing for nifi to pull them in makes sense to me and I think is
>>>>> common in container environments.
>>>>>
>>>>> Looking forward to discussing more and better approaches.
>>>>> Chad
>>>>>
>>>>>> On Mon, Oct 19, 2020 at 1:46 PM Andy LoPresto <al...@apache.org>
>>> wrote:
>>>>>>
>>>>>> Hi Chad,
>>>>>>
>>>>>> Parameters were introduced as a way to deprecate (NiFi) variables
>>>>>> entirely. I’m not sure that introducing a dependency between the two
>>> is a
>>>>>> positive step forward. I think there is a separate conversation to be
>>> had
>>>>>> about allowing parameters access to environment variables, but I
>>> think this
>>>>>> could introduce problems as parameters are designed for flexibility
>>> and
>>>>>> portability, and moving from a system where a parameter was actually a
>>>>>> pass-through to an environment variable would cause unexpected
>>> problems on
>>>>>> the destination system.
>>>>>>
>>>>>> I think the pros and cons of this need to be clearly enumerated and
>>>>>> discussed here. Thanks for bringing this up.
>>>>>>
>>>>>>
>>>>>> Andy LoPresto
>>>>>> alopresto@apache.org
>>>>>> alopresto.apache@gmail.com
>>>>>> He/Him
>>>>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
>>>>>>
>>>>>>>> On Oct 19, 2020, at 9:43 AM, Chad Zobrisky <cz...@gmail.com>
>>> wrote:
>>>>>>>
>>>>>>> Hello,
>>>>>>>
>>>>>>> I was configuring an SSL Context Controller Service today and had the
>>>>>>> keystores and passwords passed into the container via environment
>>>>>>> variables. I thought it would be nice to be able to reference these
>>> from
>>>>>>> the parameter context. Maybe either giving Parameter Context values
>>> the
>>>>>>> VARIABLE_REGISTRY scope in the Expression Language, or a new scope
>>> for
>>>>>>> references external to nifi?
>>>>>>>
>>>>>>> I think for refreshing the Parameter Context on those external
>>> changes,
>>>>>> it
>>>>>>> would require an edit/re-apply just as it does now, and would have to
>>>>>> make
>>>>>>> sure it is well documented.
>>>>>>>
>>>>>>> I'd be interested in creating a PR for this if the idea makes sense
>>> and
>>>>>> is
>>>>>>> acceptable.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Chad
>>>>>>
>>>>>>
>>>>
>>>
Re: Parameter Contexts Expression Language scopes
Posted by Bryan Bende <bb...@gmail.com>.
Sorry, the one list was meant to be...
a) system properties
b) environment variables
c) process-group variable registry
d) file-based variable registry
On Mon, Oct 19, 2020 at 4:35 PM Bryan Bende <bb...@gmail.com> wrote:
>
> I think there is some confusion based on terminology...
>
> A given property has an expression language scope defined, which can
> be "flow file attributes" or "variable registry only". This was
> created mostly for documentation purposes so that users could look at
> a property in the docs and see "expression language supported: true"
> and know which values they could reference. What it really comes down
> to in the code is the difference between
> "someProperty.evaluateAttributeExpressions()" and
> "someProperty.evaluateAttributeExpressions(flowFile)"... meaning can I
> reference a value from a flow file or not.
>
> In the case of "variable registry only", the value can come from any
> of the following...
> a) system properties
> b) expression language
> c) process-group variable registry
> d) file-based variable registry
>
> When people have stated that "variables are being deprecated in favor
> of parameters", they are referring to the last two items (c & d).
>
> The reason being that parameters solved several short-comings of those
> two options...
>
> - the ability to store sensitive values encrypted
> - the ability to reference them from any property using a new syntax
> #{...}, not long dependent on component developer saying
> "expressionLanguageSupported(true)" on the descriptor
> - the ability to create policies for which users/groups could
> reference a set of parameters
> - the hierarchical ambiguity of what "${foo}" actually resolves to
>
> If you just want to use environment variables, why use parameter
> contexts? Expression language has always offered access to env vars
> and still will going forward.
>
> The one argument I can see is that not all properties support
> expression language, so using parameters gives you a way around that.
>
>
>
>
>
> On Mon, Oct 19, 2020 at 3:58 PM Chris Sampson
> <ch...@naimuri.com.invalid> wrote:
> >
> > Being based primarily in Docker containers and having experience with both
> > Kubernetes (where secrets such as KESTORE_PASSWORD can be injected as
> > environment variables or files) and Docker Swarm (which only handles
> > secrets as environment variables), I'd have definitely been wanting this
> > before moving from Variables to Parameters if I was still in Swarm (or
> > Docker Compose/straight up Docker).
> >
> > It's certainly possible to script creating/updating Parameters via the
> > Toolkit/NiPyAPI, but in Docker Swarm that's not so easy (whereas it's
> > possible as a Job in Kubernetes, for example). So environment variables
> > could save the day in that instance.
> >
> > I guess one likely problem (but no different to how I guess the Variable
> > Registry uses env vars) would be how NiFi will handle changes to the env
> > vars - does it:
> >
> > - ignore them until instance restart, which could lead to maintainer
> > confusion (I've changed KEYSTORE_PASSWORD in the env but things are still
> > failing in NiFi)
> > - alert the maintainer to the fact that the env var has changed and a
> > Parameter needs updating, with the new value being used after all
> > associated processors/controllers have been restarted
> > - automatically attempt to update the parameters by restarting all
> > associated processors/controllers, which I'd guess would be a bit dangerous
> > for interrupting in-flow data, etc.
> >
> >
> > ---
> > *Chris Sampson*
> > IT Consultant
> > chris.sampson@naimuri.com
> > <https://www.naimuri.com/>
> >
> >
> > On Mon, 19 Oct 2020 at 19:35, Bryan Bende <bb...@gmail.com> wrote:
> >
> > > Access to environment variables directly from expression language is
> > > not being removed.
> > >
> > > The discussion is about whether a parameter value should be able to
> > > use expression language to reference an environment variable.
> > >
> > > For example, processor property has #{keystore.password} -> parameter
> > > "keystore.password" has value "${KEYSTORE_PASSWORD}" which then gets
> > > the password from an environment variable.
> > >
> > >
> > >
> > > On Mon, Oct 19, 2020 at 2:14 PM Uwe@Moosheimer.com <Uw...@moosheimer.com>
> > > wrote:
> > > >
> > > > Chad,
> > > >
> > > > So far I thought that only the NiFi variables are deprecated and access
> > > to environment variables will still be possible.
> > > >
> > > > If this is not the case, then I agree with you. It should definitely be
> > > possible to access environment variables. Otherwise I can't imagine how to
> > > refer to the hostname or the current JAVA path or ... or ... or on each
> > > node?!
> > > >
> > > > Mit freundlichen Grüßen / best regards
> > > > Kay-Uwe Moosheimer
> > > >
> > > > > Am 19.10.2020 um 20:00 schrieb Chad Zobrisky <cz...@gmail.com>:
> > > > >
> > > > > Andy,
> > > > >
> > > > > Thanks for the response!
> > > > >
> > > > > When I was thinking through this the deprecation of variables was
> > > > > definitely on my mind but the fact that it already had direct access to
> > > > > environment variables was the simplest path. I think it does make more
> > > > > sense to add access to environment variables to the parameter context,
> > > or
> > > > > allowing a specific scope just for environment variables in the
> > > > > expression language.
> > > > >
> > > > > I think giving access to environment variables actually allows more
> > > > > portability between environments, eg dev, test, prod. Defining those
> > > once
> > > > > and allowing for nifi to pull them in makes sense to me and I think is
> > > > > common in container environments.
> > > > >
> > > > > Looking forward to discussing more and better approaches.
> > > > > Chad
> > > > >
> > > > >> On Mon, Oct 19, 2020 at 1:46 PM Andy LoPresto <al...@apache.org>
> > > wrote:
> > > > >>
> > > > >> Hi Chad,
> > > > >>
> > > > >> Parameters were introduced as a way to deprecate (NiFi) variables
> > > > >> entirely. I’m not sure that introducing a dependency between the two
> > > is a
> > > > >> positive step forward. I think there is a separate conversation to be
> > > had
> > > > >> about allowing parameters access to environment variables, but I
> > > think this
> > > > >> could introduce problems as parameters are designed for flexibility
> > > and
> > > > >> portability, and moving from a system where a parameter was actually a
> > > > >> pass-through to an environment variable would cause unexpected
> > > problems on
> > > > >> the destination system.
> > > > >>
> > > > >> I think the pros and cons of this need to be clearly enumerated and
> > > > >> discussed here. Thanks for bringing this up.
> > > > >>
> > > > >>
> > > > >> Andy LoPresto
> > > > >> alopresto@apache.org
> > > > >> alopresto.apache@gmail.com
> > > > >> He/Him
> > > > >> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
> > > > >>
> > > > >>>> On Oct 19, 2020, at 9:43 AM, Chad Zobrisky <cz...@gmail.com>
> > > wrote:
> > > > >>>
> > > > >>> Hello,
> > > > >>>
> > > > >>> I was configuring an SSL Context Controller Service today and had the
> > > > >>> keystores and passwords passed into the container via environment
> > > > >>> variables. I thought it would be nice to be able to reference these
> > > from
> > > > >>> the parameter context. Maybe either giving Parameter Context values
> > > the
> > > > >>> VARIABLE_REGISTRY scope in the Expression Language, or a new scope
> > > for
> > > > >>> references external to nifi?
> > > > >>>
> > > > >>> I think for refreshing the Parameter Context on those external
> > > changes,
> > > > >> it
> > > > >>> would require an edit/re-apply just as it does now, and would have to
> > > > >> make
> > > > >>> sure it is well documented.
> > > > >>>
> > > > >>> I'd be interested in creating a PR for this if the idea makes sense
> > > and
> > > > >> is
> > > > >>> acceptable.
> > > > >>>
> > > > >>> Thanks,
> > > > >>> Chad
> > > > >>
> > > > >>
> > > >
> > >
Re: Parameter Contexts Expression Language scopes
Posted by Bryan Bende <bb...@gmail.com>.
I think there is some confusion based on terminology...
A given property has an expression language scope defined, which can
be "flow file attributes" or "variable registry only". This was
created mostly for documentation purposes so that users could look at
a property in the docs and see "expression language supported: true"
and know which values they could reference. What it really comes down
to in the code is the difference between
"someProperty.evaluateAttributeExpressions()" and
"someProperty.evaluateAttributeExpressions(flowFile)"... meaning can I
reference a value from a flow file or not.
In the case of "variable registry only", the value can come from any
of the following...
a) system properties
b) expression language
c) process-group variable registry
d) file-based variable registry
When people have stated that "variables are being deprecated in favor
of parameters", they are referring to the last two items (c & d).
The reason being that parameters solved several short-comings of those
two options...
- the ability to store sensitive values encrypted
- the ability to reference them from any property using a new syntax
#{...}, not long dependent on component developer saying
"expressionLanguageSupported(true)" on the descriptor
- the ability to create policies for which users/groups could
reference a set of parameters
- the hierarchical ambiguity of what "${foo}" actually resolves to
If you just want to use environment variables, why use parameter
contexts? Expression language has always offered access to env vars
and still will going forward.
The one argument I can see is that not all properties support
expression language, so using parameters gives you a way around that.
On Mon, Oct 19, 2020 at 3:58 PM Chris Sampson
<ch...@naimuri.com.invalid> wrote:
>
> Being based primarily in Docker containers and having experience with both
> Kubernetes (where secrets such as KESTORE_PASSWORD can be injected as
> environment variables or files) and Docker Swarm (which only handles
> secrets as environment variables), I'd have definitely been wanting this
> before moving from Variables to Parameters if I was still in Swarm (or
> Docker Compose/straight up Docker).
>
> It's certainly possible to script creating/updating Parameters via the
> Toolkit/NiPyAPI, but in Docker Swarm that's not so easy (whereas it's
> possible as a Job in Kubernetes, for example). So environment variables
> could save the day in that instance.
>
> I guess one likely problem (but no different to how I guess the Variable
> Registry uses env vars) would be how NiFi will handle changes to the env
> vars - does it:
>
> - ignore them until instance restart, which could lead to maintainer
> confusion (I've changed KEYSTORE_PASSWORD in the env but things are still
> failing in NiFi)
> - alert the maintainer to the fact that the env var has changed and a
> Parameter needs updating, with the new value being used after all
> associated processors/controllers have been restarted
> - automatically attempt to update the parameters by restarting all
> associated processors/controllers, which I'd guess would be a bit dangerous
> for interrupting in-flow data, etc.
>
>
> ---
> *Chris Sampson*
> IT Consultant
> chris.sampson@naimuri.com
> <https://www.naimuri.com/>
>
>
> On Mon, 19 Oct 2020 at 19:35, Bryan Bende <bb...@gmail.com> wrote:
>
> > Access to environment variables directly from expression language is
> > not being removed.
> >
> > The discussion is about whether a parameter value should be able to
> > use expression language to reference an environment variable.
> >
> > For example, processor property has #{keystore.password} -> parameter
> > "keystore.password" has value "${KEYSTORE_PASSWORD}" which then gets
> > the password from an environment variable.
> >
> >
> >
> > On Mon, Oct 19, 2020 at 2:14 PM Uwe@Moosheimer.com <Uw...@moosheimer.com>
> > wrote:
> > >
> > > Chad,
> > >
> > > So far I thought that only the NiFi variables are deprecated and access
> > to environment variables will still be possible.
> > >
> > > If this is not the case, then I agree with you. It should definitely be
> > possible to access environment variables. Otherwise I can't imagine how to
> > refer to the hostname or the current JAVA path or ... or ... or on each
> > node?!
> > >
> > > Mit freundlichen Grüßen / best regards
> > > Kay-Uwe Moosheimer
> > >
> > > > Am 19.10.2020 um 20:00 schrieb Chad Zobrisky <cz...@gmail.com>:
> > > >
> > > > Andy,
> > > >
> > > > Thanks for the response!
> > > >
> > > > When I was thinking through this the deprecation of variables was
> > > > definitely on my mind but the fact that it already had direct access to
> > > > environment variables was the simplest path. I think it does make more
> > > > sense to add access to environment variables to the parameter context,
> > or
> > > > allowing a specific scope just for environment variables in the
> > > > expression language.
> > > >
> > > > I think giving access to environment variables actually allows more
> > > > portability between environments, eg dev, test, prod. Defining those
> > once
> > > > and allowing for nifi to pull them in makes sense to me and I think is
> > > > common in container environments.
> > > >
> > > > Looking forward to discussing more and better approaches.
> > > > Chad
> > > >
> > > >> On Mon, Oct 19, 2020 at 1:46 PM Andy LoPresto <al...@apache.org>
> > wrote:
> > > >>
> > > >> Hi Chad,
> > > >>
> > > >> Parameters were introduced as a way to deprecate (NiFi) variables
> > > >> entirely. I’m not sure that introducing a dependency between the two
> > is a
> > > >> positive step forward. I think there is a separate conversation to be
> > had
> > > >> about allowing parameters access to environment variables, but I
> > think this
> > > >> could introduce problems as parameters are designed for flexibility
> > and
> > > >> portability, and moving from a system where a parameter was actually a
> > > >> pass-through to an environment variable would cause unexpected
> > problems on
> > > >> the destination system.
> > > >>
> > > >> I think the pros and cons of this need to be clearly enumerated and
> > > >> discussed here. Thanks for bringing this up.
> > > >>
> > > >>
> > > >> Andy LoPresto
> > > >> alopresto@apache.org
> > > >> alopresto.apache@gmail.com
> > > >> He/Him
> > > >> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
> > > >>
> > > >>>> On Oct 19, 2020, at 9:43 AM, Chad Zobrisky <cz...@gmail.com>
> > wrote:
> > > >>>
> > > >>> Hello,
> > > >>>
> > > >>> I was configuring an SSL Context Controller Service today and had the
> > > >>> keystores and passwords passed into the container via environment
> > > >>> variables. I thought it would be nice to be able to reference these
> > from
> > > >>> the parameter context. Maybe either giving Parameter Context values
> > the
> > > >>> VARIABLE_REGISTRY scope in the Expression Language, or a new scope
> > for
> > > >>> references external to nifi?
> > > >>>
> > > >>> I think for refreshing the Parameter Context on those external
> > changes,
> > > >> it
> > > >>> would require an edit/re-apply just as it does now, and would have to
> > > >> make
> > > >>> sure it is well documented.
> > > >>>
> > > >>> I'd be interested in creating a PR for this if the idea makes sense
> > and
> > > >> is
> > > >>> acceptable.
> > > >>>
> > > >>> Thanks,
> > > >>> Chad
> > > >>
> > > >>
> > >
> >
Re: Parameter Contexts Expression Language scopes
Posted by Chris Sampson <ch...@naimuri.com.INVALID>.
Being based primarily in Docker containers and having experience with both
Kubernetes (where secrets such as KESTORE_PASSWORD can be injected as
environment variables or files) and Docker Swarm (which only handles
secrets as environment variables), I'd have definitely been wanting this
before moving from Variables to Parameters if I was still in Swarm (or
Docker Compose/straight up Docker).
It's certainly possible to script creating/updating Parameters via the
Toolkit/NiPyAPI, but in Docker Swarm that's not so easy (whereas it's
possible as a Job in Kubernetes, for example). So environment variables
could save the day in that instance.
I guess one likely problem (but no different to how I guess the Variable
Registry uses env vars) would be how NiFi will handle changes to the env
vars - does it:
- ignore them until instance restart, which could lead to maintainer
confusion (I've changed KEYSTORE_PASSWORD in the env but things are still
failing in NiFi)
- alert the maintainer to the fact that the env var has changed and a
Parameter needs updating, with the new value being used after all
associated processors/controllers have been restarted
- automatically attempt to update the parameters by restarting all
associated processors/controllers, which I'd guess would be a bit dangerous
for interrupting in-flow data, etc.
---
*Chris Sampson*
IT Consultant
chris.sampson@naimuri.com
<https://www.naimuri.com/>
On Mon, 19 Oct 2020 at 19:35, Bryan Bende <bb...@gmail.com> wrote:
> Access to environment variables directly from expression language is
> not being removed.
>
> The discussion is about whether a parameter value should be able to
> use expression language to reference an environment variable.
>
> For example, processor property has #{keystore.password} -> parameter
> "keystore.password" has value "${KEYSTORE_PASSWORD}" which then gets
> the password from an environment variable.
>
>
>
> On Mon, Oct 19, 2020 at 2:14 PM Uwe@Moosheimer.com <Uw...@moosheimer.com>
> wrote:
> >
> > Chad,
> >
> > So far I thought that only the NiFi variables are deprecated and access
> to environment variables will still be possible.
> >
> > If this is not the case, then I agree with you. It should definitely be
> possible to access environment variables. Otherwise I can't imagine how to
> refer to the hostname or the current JAVA path or ... or ... or on each
> node?!
> >
> > Mit freundlichen Grüßen / best regards
> > Kay-Uwe Moosheimer
> >
> > > Am 19.10.2020 um 20:00 schrieb Chad Zobrisky <cz...@gmail.com>:
> > >
> > > Andy,
> > >
> > > Thanks for the response!
> > >
> > > When I was thinking through this the deprecation of variables was
> > > definitely on my mind but the fact that it already had direct access to
> > > environment variables was the simplest path. I think it does make more
> > > sense to add access to environment variables to the parameter context,
> or
> > > allowing a specific scope just for environment variables in the
> > > expression language.
> > >
> > > I think giving access to environment variables actually allows more
> > > portability between environments, eg dev, test, prod. Defining those
> once
> > > and allowing for nifi to pull them in makes sense to me and I think is
> > > common in container environments.
> > >
> > > Looking forward to discussing more and better approaches.
> > > Chad
> > >
> > >> On Mon, Oct 19, 2020 at 1:46 PM Andy LoPresto <al...@apache.org>
> wrote:
> > >>
> > >> Hi Chad,
> > >>
> > >> Parameters were introduced as a way to deprecate (NiFi) variables
> > >> entirely. I’m not sure that introducing a dependency between the two
> is a
> > >> positive step forward. I think there is a separate conversation to be
> had
> > >> about allowing parameters access to environment variables, but I
> think this
> > >> could introduce problems as parameters are designed for flexibility
> and
> > >> portability, and moving from a system where a parameter was actually a
> > >> pass-through to an environment variable would cause unexpected
> problems on
> > >> the destination system.
> > >>
> > >> I think the pros and cons of this need to be clearly enumerated and
> > >> discussed here. Thanks for bringing this up.
> > >>
> > >>
> > >> Andy LoPresto
> > >> alopresto@apache.org
> > >> alopresto.apache@gmail.com
> > >> He/Him
> > >> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
> > >>
> > >>>> On Oct 19, 2020, at 9:43 AM, Chad Zobrisky <cz...@gmail.com>
> wrote:
> > >>>
> > >>> Hello,
> > >>>
> > >>> I was configuring an SSL Context Controller Service today and had the
> > >>> keystores and passwords passed into the container via environment
> > >>> variables. I thought it would be nice to be able to reference these
> from
> > >>> the parameter context. Maybe either giving Parameter Context values
> the
> > >>> VARIABLE_REGISTRY scope in the Expression Language, or a new scope
> for
> > >>> references external to nifi?
> > >>>
> > >>> I think for refreshing the Parameter Context on those external
> changes,
> > >> it
> > >>> would require an edit/re-apply just as it does now, and would have to
> > >> make
> > >>> sure it is well documented.
> > >>>
> > >>> I'd be interested in creating a PR for this if the idea makes sense
> and
> > >> is
> > >>> acceptable.
> > >>>
> > >>> Thanks,
> > >>> Chad
> > >>
> > >>
> >
>
Re: Parameter Contexts Expression Language scopes
Posted by Bryan Bende <bb...@gmail.com>.
Access to environment variables directly from expression language is
not being removed.
The discussion is about whether a parameter value should be able to
use expression language to reference an environment variable.
For example, processor property has #{keystore.password} -> parameter
"keystore.password" has value "${KEYSTORE_PASSWORD}" which then gets
the password from an environment variable.
On Mon, Oct 19, 2020 at 2:14 PM Uwe@Moosheimer.com <Uw...@moosheimer.com> wrote:
>
> Chad,
>
> So far I thought that only the NiFi variables are deprecated and access to environment variables will still be possible.
>
> If this is not the case, then I agree with you. It should definitely be possible to access environment variables. Otherwise I can't imagine how to refer to the hostname or the current JAVA path or ... or ... or on each node?!
>
> Mit freundlichen Grüßen / best regards
> Kay-Uwe Moosheimer
>
> > Am 19.10.2020 um 20:00 schrieb Chad Zobrisky <cz...@gmail.com>:
> >
> > Andy,
> >
> > Thanks for the response!
> >
> > When I was thinking through this the deprecation of variables was
> > definitely on my mind but the fact that it already had direct access to
> > environment variables was the simplest path. I think it does make more
> > sense to add access to environment variables to the parameter context, or
> > allowing a specific scope just for environment variables in the
> > expression language.
> >
> > I think giving access to environment variables actually allows more
> > portability between environments, eg dev, test, prod. Defining those once
> > and allowing for nifi to pull them in makes sense to me and I think is
> > common in container environments.
> >
> > Looking forward to discussing more and better approaches.
> > Chad
> >
> >> On Mon, Oct 19, 2020 at 1:46 PM Andy LoPresto <al...@apache.org> wrote:
> >>
> >> Hi Chad,
> >>
> >> Parameters were introduced as a way to deprecate (NiFi) variables
> >> entirely. I’m not sure that introducing a dependency between the two is a
> >> positive step forward. I think there is a separate conversation to be had
> >> about allowing parameters access to environment variables, but I think this
> >> could introduce problems as parameters are designed for flexibility and
> >> portability, and moving from a system where a parameter was actually a
> >> pass-through to an environment variable would cause unexpected problems on
> >> the destination system.
> >>
> >> I think the pros and cons of this need to be clearly enumerated and
> >> discussed here. Thanks for bringing this up.
> >>
> >>
> >> Andy LoPresto
> >> alopresto@apache.org
> >> alopresto.apache@gmail.com
> >> He/Him
> >> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
> >>
> >>>> On Oct 19, 2020, at 9:43 AM, Chad Zobrisky <cz...@gmail.com> wrote:
> >>>
> >>> Hello,
> >>>
> >>> I was configuring an SSL Context Controller Service today and had the
> >>> keystores and passwords passed into the container via environment
> >>> variables. I thought it would be nice to be able to reference these from
> >>> the parameter context. Maybe either giving Parameter Context values the
> >>> VARIABLE_REGISTRY scope in the Expression Language, or a new scope for
> >>> references external to nifi?
> >>>
> >>> I think for refreshing the Parameter Context on those external changes,
> >> it
> >>> would require an edit/re-apply just as it does now, and would have to
> >> make
> >>> sure it is well documented.
> >>>
> >>> I'd be interested in creating a PR for this if the idea makes sense and
> >> is
> >>> acceptable.
> >>>
> >>> Thanks,
> >>> Chad
> >>
> >>
>
Re: Parameter Contexts Expression Language scopes
Posted by "Uwe@Moosheimer.com" <Uw...@moosheimer.com>.
Chad,
So far I thought that only the NiFi variables are deprecated and access to environment variables will still be possible.
If this is not the case, then I agree with you. It should definitely be possible to access environment variables. Otherwise I can't imagine how to refer to the hostname or the current JAVA path or ... or ... or on each node?!
Mit freundlichen Grüßen / best regards
Kay-Uwe Moosheimer
> Am 19.10.2020 um 20:00 schrieb Chad Zobrisky <cz...@gmail.com>:
>
> Andy,
>
> Thanks for the response!
>
> When I was thinking through this the deprecation of variables was
> definitely on my mind but the fact that it already had direct access to
> environment variables was the simplest path. I think it does make more
> sense to add access to environment variables to the parameter context, or
> allowing a specific scope just for environment variables in the
> expression language.
>
> I think giving access to environment variables actually allows more
> portability between environments, eg dev, test, prod. Defining those once
> and allowing for nifi to pull them in makes sense to me and I think is
> common in container environments.
>
> Looking forward to discussing more and better approaches.
> Chad
>
>> On Mon, Oct 19, 2020 at 1:46 PM Andy LoPresto <al...@apache.org> wrote:
>>
>> Hi Chad,
>>
>> Parameters were introduced as a way to deprecate (NiFi) variables
>> entirely. I’m not sure that introducing a dependency between the two is a
>> positive step forward. I think there is a separate conversation to be had
>> about allowing parameters access to environment variables, but I think this
>> could introduce problems as parameters are designed for flexibility and
>> portability, and moving from a system where a parameter was actually a
>> pass-through to an environment variable would cause unexpected problems on
>> the destination system.
>>
>> I think the pros and cons of this need to be clearly enumerated and
>> discussed here. Thanks for bringing this up.
>>
>>
>> Andy LoPresto
>> alopresto@apache.org
>> alopresto.apache@gmail.com
>> He/Him
>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
>>
>>>> On Oct 19, 2020, at 9:43 AM, Chad Zobrisky <cz...@gmail.com> wrote:
>>>
>>> Hello,
>>>
>>> I was configuring an SSL Context Controller Service today and had the
>>> keystores and passwords passed into the container via environment
>>> variables. I thought it would be nice to be able to reference these from
>>> the parameter context. Maybe either giving Parameter Context values the
>>> VARIABLE_REGISTRY scope in the Expression Language, or a new scope for
>>> references external to nifi?
>>>
>>> I think for refreshing the Parameter Context on those external changes,
>> it
>>> would require an edit/re-apply just as it does now, and would have to
>> make
>>> sure it is well documented.
>>>
>>> I'd be interested in creating a PR for this if the idea makes sense and
>> is
>>> acceptable.
>>>
>>> Thanks,
>>> Chad
>>
>>
Re: Parameter Contexts Expression Language scopes
Posted by Chad Zobrisky <cz...@gmail.com>.
Andy,
Thanks for the response!
When I was thinking through this the deprecation of variables was
definitely on my mind but the fact that it already had direct access to
environment variables was the simplest path. I think it does make more
sense to add access to environment variables to the parameter context, or
allowing a specific scope just for environment variables in the
expression language.
I think giving access to environment variables actually allows more
portability between environments, eg dev, test, prod. Defining those once
and allowing for nifi to pull them in makes sense to me and I think is
common in container environments.
Looking forward to discussing more and better approaches.
Chad
On Mon, Oct 19, 2020 at 1:46 PM Andy LoPresto <al...@apache.org> wrote:
> Hi Chad,
>
> Parameters were introduced as a way to deprecate (NiFi) variables
> entirely. I’m not sure that introducing a dependency between the two is a
> positive step forward. I think there is a separate conversation to be had
> about allowing parameters access to environment variables, but I think this
> could introduce problems as parameters are designed for flexibility and
> portability, and moving from a system where a parameter was actually a
> pass-through to an environment variable would cause unexpected problems on
> the destination system.
>
> I think the pros and cons of this need to be clearly enumerated and
> discussed here. Thanks for bringing this up.
>
>
> Andy LoPresto
> alopresto@apache.org
> alopresto.apache@gmail.com
> He/Him
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
>
> > On Oct 19, 2020, at 9:43 AM, Chad Zobrisky <cz...@gmail.com> wrote:
> >
> > Hello,
> >
> > I was configuring an SSL Context Controller Service today and had the
> > keystores and passwords passed into the container via environment
> > variables. I thought it would be nice to be able to reference these from
> > the parameter context. Maybe either giving Parameter Context values the
> > VARIABLE_REGISTRY scope in the Expression Language, or a new scope for
> > references external to nifi?
> >
> > I think for refreshing the Parameter Context on those external changes,
> it
> > would require an edit/re-apply just as it does now, and would have to
> make
> > sure it is well documented.
> >
> > I'd be interested in creating a PR for this if the idea makes sense and
> is
> > acceptable.
> >
> > Thanks,
> > Chad
>
>
Re: Parameter Contexts Expression Language scopes
Posted by Andy LoPresto <al...@apache.org>.
Hi Chad,
Parameters were introduced as a way to deprecate (NiFi) variables entirely. I’m not sure that introducing a dependency between the two is a positive step forward. I think there is a separate conversation to be had about allowing parameters access to environment variables, but I think this could introduce problems as parameters are designed for flexibility and portability, and moving from a system where a parameter was actually a pass-through to an environment variable would cause unexpected problems on the destination system.
I think the pros and cons of this need to be clearly enumerated and discussed here. Thanks for bringing this up.
Andy LoPresto
alopresto@apache.org
alopresto.apache@gmail.com
He/Him
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
> On Oct 19, 2020, at 9:43 AM, Chad Zobrisky <cz...@gmail.com> wrote:
>
> Hello,
>
> I was configuring an SSL Context Controller Service today and had the
> keystores and passwords passed into the container via environment
> variables. I thought it would be nice to be able to reference these from
> the parameter context. Maybe either giving Parameter Context values the
> VARIABLE_REGISTRY scope in the Expression Language, or a new scope for
> references external to nifi?
>
> I think for refreshing the Parameter Context on those external changes, it
> would require an edit/re-apply just as it does now, and would have to make
> sure it is well documented.
>
> I'd be interested in creating a PR for this if the idea makes sense and is
> acceptable.
>
> Thanks,
> Chad