You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@felix.apache.org by David Bosschaert <da...@gmail.com> on 2019/07/25 14:32:33 UTC
Re: Potential Felix contribution: ConfigAdmin plugin that can
substitute variable placeholders (e.g. for K8s secrets)
Hi all,
Picking this thread up after some time.
I'll start contributing the initial code for this config admin plugin as a
new maven module soon. We can expand it then.
I was thinking of putting it at configadmin-plugins/substitution
Best regards,
David
On Tue, 11 Jun 2019 at 17:11, Jean-Baptiste Onofré <jb...@nanthrax.net> wrote:
> Cool, I would be more than happy to work with you on this one !
>
> Regards
> JB
>
> On 11/06/2019 16:14, David Bosschaert wrote:
> > Hi JB,
> >
> > On Tue, 11 Jun 2019 at 14:57, Jean-Baptiste Onofré <jb...@nanthrax.net>
> wrote:
> >
> >> It sounds interesting.
> >>
> >> Does it also support replacement with "system variables" (like
> >> -Duser=my-user) ?
> >>
> >> I started a ConfigAdmin "hack" for Karaf about that, especially when
> >> running in Docker.
> >>
> >>
> > Not yet. But it could certainly be expanded to do this.
> >
> > Cheers,
> >
> > David
> >
>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>
Re: Potential Felix contribution: ConfigAdmin plugin that can
substitute variable placeholders (e.g. for K8s secrets)
Posted by Raymond Auge <ra...@liferay.com>.
👍
On Fri, Jul 26, 2019 at 10:20 AM David Bosschaert <
david.bosschaert@gmail.com> wrote:
> While Carsten reviewed the code earlier today he suggested that this
> mechanism be supported, so the plugin is registered with a
> config.plugin.id
> value of 'org.apache.felix.configadmin.plugin.interpolation' [1]
>
> Thanks for the review, Carsten!
>
> David
>
> [1]
>
> https://github.com/apache/felix/blob/trunk/configadmin-plugins/interpolation/src/main/java/org/apache/felix/configadmin/plugin/interpolation/Activator.java#L44
>
> On Fri, 26 Jul 2019 at 15:02, Raymond Auge <ra...@liferay.com>
> wrote:
>
> > Perfect (I didn't look at the impl), but as long as there's a way to
> > enforce it! :)
> >
> > - Ray
> >
> > On Fri, Jul 26, 2019 at 9:58 AM Carsten Ziegeler <cz...@apache.org>
> > wrote:
> >
> > > Not sure, if that's what you're looking for, but with FELIX-6059 you
> can
> > > configure the configuration admin to depend on the plugin being
> available
> > >
> > > Carsten
> > >
> > > Raymond Auge wrote
> > > > An idea for the future might be a requirement so that we can make
> sure
> > > the
> > > > feature is available during provisioning of the framework bundles.
> > > >
> > > > - Ray
> > > >
> > > > On Fri, Jul 26, 2019 at 7:12 AM David Bosschaert <
> > > david.bosschaert@gmail.com>
> > > > wrote:
> > > >
> > > >> Thanks for the suggestion, Bertrand!
> > > >>
> > > >> I have renamed the plugin to 'interpolation'.
> > > >> I have also implemented framework/system property based substitution
> > as
> > > >> suggested by JB Onofré
> > > >> And I have added substitution support for environment variables,
> which
> > > can
> > > >> be useful when you'd like to configure your runtime (e.g.
> containers)
> > > >> 12-factor style
> > > >>
> > > >> The code is here:
> > > >>
> > > >>
> > >
> >
> https://svn.apache.org/repos/asf/felix/trunk/configadmin-plugins/interpolation
> > > >> Documentation is best read on github:
> > > >>
> > > >>
> > >
> >
> https://github.com/apache/felix/blob/trunk/configadmin-plugins/interpolation/README.md
> > > >>
> > > >> I'd like to do an initial 0.0.2 release early next week to make it
> > easy
> > > for
> > > >> everyone to try it out.
> > > >>
> > > >> Any suggestions, let me know!
> > > >>
> > > >> Cheers,
> > > >>
> > > >> David
> > > >>
> > > >>
> > > >> On Fri, 26 Jul 2019 at 10:06, Bertrand Delacretaz <
> > > bdelacretaz@apache.org>
> > > >> wrote:
> > > >>
> > > >>> Hi,
> > > >>>
> > > >>> On Thu, Jul 25, 2019 at 4:32 PM David Bosschaert
> > > >>> <da...@gmail.com> wrote:
> > > >>>> ...I was thinking of putting it at
> > configadmin-plugins/substitution..
> > > >>>
> > > >>> FWIW, "interpolation" is a common term for that as per
> > > >>> https://en.wikipedia.org/wiki/String_interpolation
> > > >>>
> > > >>> -Bertrand
> > > >>>
> > > >>
> > > >
> > > >
> > > --
> > > Carsten Ziegeler
> > > Adobe Research Switzerland
> > > cziegeler@apache.org
> > >
> >
> >
> > --
> > *Raymond Augé* <http://www.liferay.com/web/raymond.auge/profile>
> > (@rotty3000)
> > Senior Software Architect *Liferay, Inc.* <http://www.liferay.com>
> > (@Liferay)
> > Board Member & EEG Co-Chair, OSGi Alliance <http://osgi.org>
> > (@OSGiAlliance)
> >
>
--
*Raymond Augé* <http://www.liferay.com/web/raymond.auge/profile>
(@rotty3000)
Senior Software Architect *Liferay, Inc.* <http://www.liferay.com>
(@Liferay)
Board Member & EEG Co-Chair, OSGi Alliance <http://osgi.org> (@OSGiAlliance)
Re: Potential Felix contribution: ConfigAdmin plugin that can
substitute variable placeholders (e.g. for K8s secrets)
Posted by David Bosschaert <da...@gmail.com>.
While Carsten reviewed the code earlier today he suggested that this
mechanism be supported, so the plugin is registered with a config.plugin.id
value of 'org.apache.felix.configadmin.plugin.interpolation' [1]
Thanks for the review, Carsten!
David
[1]
https://github.com/apache/felix/blob/trunk/configadmin-plugins/interpolation/src/main/java/org/apache/felix/configadmin/plugin/interpolation/Activator.java#L44
On Fri, 26 Jul 2019 at 15:02, Raymond Auge <ra...@liferay.com> wrote:
> Perfect (I didn't look at the impl), but as long as there's a way to
> enforce it! :)
>
> - Ray
>
> On Fri, Jul 26, 2019 at 9:58 AM Carsten Ziegeler <cz...@apache.org>
> wrote:
>
> > Not sure, if that's what you're looking for, but with FELIX-6059 you can
> > configure the configuration admin to depend on the plugin being available
> >
> > Carsten
> >
> > Raymond Auge wrote
> > > An idea for the future might be a requirement so that we can make sure
> > the
> > > feature is available during provisioning of the framework bundles.
> > >
> > > - Ray
> > >
> > > On Fri, Jul 26, 2019 at 7:12 AM David Bosschaert <
> > david.bosschaert@gmail.com>
> > > wrote:
> > >
> > >> Thanks for the suggestion, Bertrand!
> > >>
> > >> I have renamed the plugin to 'interpolation'.
> > >> I have also implemented framework/system property based substitution
> as
> > >> suggested by JB Onofré
> > >> And I have added substitution support for environment variables, which
> > can
> > >> be useful when you'd like to configure your runtime (e.g. containers)
> > >> 12-factor style
> > >>
> > >> The code is here:
> > >>
> > >>
> >
> https://svn.apache.org/repos/asf/felix/trunk/configadmin-plugins/interpolation
> > >> Documentation is best read on github:
> > >>
> > >>
> >
> https://github.com/apache/felix/blob/trunk/configadmin-plugins/interpolation/README.md
> > >>
> > >> I'd like to do an initial 0.0.2 release early next week to make it
> easy
> > for
> > >> everyone to try it out.
> > >>
> > >> Any suggestions, let me know!
> > >>
> > >> Cheers,
> > >>
> > >> David
> > >>
> > >>
> > >> On Fri, 26 Jul 2019 at 10:06, Bertrand Delacretaz <
> > bdelacretaz@apache.org>
> > >> wrote:
> > >>
> > >>> Hi,
> > >>>
> > >>> On Thu, Jul 25, 2019 at 4:32 PM David Bosschaert
> > >>> <da...@gmail.com> wrote:
> > >>>> ...I was thinking of putting it at
> configadmin-plugins/substitution..
> > >>>
> > >>> FWIW, "interpolation" is a common term for that as per
> > >>> https://en.wikipedia.org/wiki/String_interpolation
> > >>>
> > >>> -Bertrand
> > >>>
> > >>
> > >
> > >
> > --
> > Carsten Ziegeler
> > Adobe Research Switzerland
> > cziegeler@apache.org
> >
>
>
> --
> *Raymond Augé* <http://www.liferay.com/web/raymond.auge/profile>
> (@rotty3000)
> Senior Software Architect *Liferay, Inc.* <http://www.liferay.com>
> (@Liferay)
> Board Member & EEG Co-Chair, OSGi Alliance <http://osgi.org>
> (@OSGiAlliance)
>
Re: Potential Felix contribution: ConfigAdmin plugin that can
substitute variable placeholders (e.g. for K8s secrets)
Posted by Raymond Auge <ra...@liferay.com>.
Perfect (I didn't look at the impl), but as long as there's a way to
enforce it! :)
- Ray
On Fri, Jul 26, 2019 at 9:58 AM Carsten Ziegeler <cz...@apache.org>
wrote:
> Not sure, if that's what you're looking for, but with FELIX-6059 you can
> configure the configuration admin to depend on the plugin being available
>
> Carsten
>
> Raymond Auge wrote
> > An idea for the future might be a requirement so that we can make sure
> the
> > feature is available during provisioning of the framework bundles.
> >
> > - Ray
> >
> > On Fri, Jul 26, 2019 at 7:12 AM David Bosschaert <
> david.bosschaert@gmail.com>
> > wrote:
> >
> >> Thanks for the suggestion, Bertrand!
> >>
> >> I have renamed the plugin to 'interpolation'.
> >> I have also implemented framework/system property based substitution as
> >> suggested by JB Onofré
> >> And I have added substitution support for environment variables, which
> can
> >> be useful when you'd like to configure your runtime (e.g. containers)
> >> 12-factor style
> >>
> >> The code is here:
> >>
> >>
> https://svn.apache.org/repos/asf/felix/trunk/configadmin-plugins/interpolation
> >> Documentation is best read on github:
> >>
> >>
> https://github.com/apache/felix/blob/trunk/configadmin-plugins/interpolation/README.md
> >>
> >> I'd like to do an initial 0.0.2 release early next week to make it easy
> for
> >> everyone to try it out.
> >>
> >> Any suggestions, let me know!
> >>
> >> Cheers,
> >>
> >> David
> >>
> >>
> >> On Fri, 26 Jul 2019 at 10:06, Bertrand Delacretaz <
> bdelacretaz@apache.org>
> >> wrote:
> >>
> >>> Hi,
> >>>
> >>> On Thu, Jul 25, 2019 at 4:32 PM David Bosschaert
> >>> <da...@gmail.com> wrote:
> >>>> ...I was thinking of putting it at configadmin-plugins/substitution..
> >>>
> >>> FWIW, "interpolation" is a common term for that as per
> >>> https://en.wikipedia.org/wiki/String_interpolation
> >>>
> >>> -Bertrand
> >>>
> >>
> >
> >
> --
> Carsten Ziegeler
> Adobe Research Switzerland
> cziegeler@apache.org
>
--
*Raymond Augé* <http://www.liferay.com/web/raymond.auge/profile>
(@rotty3000)
Senior Software Architect *Liferay, Inc.* <http://www.liferay.com>
(@Liferay)
Board Member & EEG Co-Chair, OSGi Alliance <http://osgi.org> (@OSGiAlliance)
Re: Potential Felix contribution: ConfigAdmin plugin that can
substitute variable placeholders (e.g. for K8s secrets)
Posted by Carsten Ziegeler <cz...@apache.org>.
Not sure, if that's what you're looking for, but with FELIX-6059 you can
configure the configuration admin to depend on the plugin being available
Carsten
Raymond Auge wrote
> An idea for the future might be a requirement so that we can make sure the
> feature is available during provisioning of the framework bundles.
>
> - Ray
>
> On Fri, Jul 26, 2019 at 7:12 AM David Bosschaert <da...@gmail.com>
> wrote:
>
>> Thanks for the suggestion, Bertrand!
>>
>> I have renamed the plugin to 'interpolation'.
>> I have also implemented framework/system property based substitution as
>> suggested by JB Onofré
>> And I have added substitution support for environment variables, which can
>> be useful when you'd like to configure your runtime (e.g. containers)
>> 12-factor style
>>
>> The code is here:
>>
>> https://svn.apache.org/repos/asf/felix/trunk/configadmin-plugins/interpolation
>> Documentation is best read on github:
>>
>> https://github.com/apache/felix/blob/trunk/configadmin-plugins/interpolation/README.md
>>
>> I'd like to do an initial 0.0.2 release early next week to make it easy for
>> everyone to try it out.
>>
>> Any suggestions, let me know!
>>
>> Cheers,
>>
>> David
>>
>>
>> On Fri, 26 Jul 2019 at 10:06, Bertrand Delacretaz <bd...@apache.org>
>> wrote:
>>
>>> Hi,
>>>
>>> On Thu, Jul 25, 2019 at 4:32 PM David Bosschaert
>>> <da...@gmail.com> wrote:
>>>> ...I was thinking of putting it at configadmin-plugins/substitution..
>>>
>>> FWIW, "interpolation" is a common term for that as per
>>> https://en.wikipedia.org/wiki/String_interpolation
>>>
>>> -Bertrand
>>>
>>
>
>
--
Carsten Ziegeler
Adobe Research Switzerland
cziegeler@apache.org
Re: Potential Felix contribution: ConfigAdmin plugin that can
substitute variable placeholders (e.g. for K8s secrets)
Posted by Raymond Auge <ra...@liferay.com>.
An idea for the future might be a requirement so that we can make sure the
feature is available during provisioning of the framework bundles.
- Ray
On Fri, Jul 26, 2019 at 7:12 AM David Bosschaert <da...@gmail.com>
wrote:
> Thanks for the suggestion, Bertrand!
>
> I have renamed the plugin to 'interpolation'.
> I have also implemented framework/system property based substitution as
> suggested by JB Onofré
> And I have added substitution support for environment variables, which can
> be useful when you'd like to configure your runtime (e.g. containers)
> 12-factor style
>
> The code is here:
>
> https://svn.apache.org/repos/asf/felix/trunk/configadmin-plugins/interpolation
> Documentation is best read on github:
>
> https://github.com/apache/felix/blob/trunk/configadmin-plugins/interpolation/README.md
>
> I'd like to do an initial 0.0.2 release early next week to make it easy for
> everyone to try it out.
>
> Any suggestions, let me know!
>
> Cheers,
>
> David
>
>
> On Fri, 26 Jul 2019 at 10:06, Bertrand Delacretaz <bd...@apache.org>
> wrote:
>
> > Hi,
> >
> > On Thu, Jul 25, 2019 at 4:32 PM David Bosschaert
> > <da...@gmail.com> wrote:
> > > ...I was thinking of putting it at configadmin-plugins/substitution..
> >
> > FWIW, "interpolation" is a common term for that as per
> > https://en.wikipedia.org/wiki/String_interpolation
> >
> > -Bertrand
> >
>
--
*Raymond Augé* <http://www.liferay.com/web/raymond.auge/profile>
(@rotty3000)
Senior Software Architect *Liferay, Inc.* <http://www.liferay.com>
(@Liferay)
Board Member & EEG Co-Chair, OSGi Alliance <http://osgi.org> (@OSGiAlliance)
Re: Potential Felix contribution: ConfigAdmin plugin that can
substitute variable placeholders (e.g. for K8s secrets)
Posted by David Bosschaert <da...@gmail.com>.
Hi all,
For the renaming of the configuration property I created:
https://issues.apache.org/jira/browse/FELIX-6165
If a property cannot be replaced, it is already left as-is, with
placeholders brackets and all. I added an extra test to show this:
https://github.com/apache/felix/blob/trunk/configadmin-plugins/interpolation/src/test/java/org/apache/felix/configadmin/plugin/interpolation/InterpolationConfigurationPluginTest.java#L90
Best regards,
David
On Wed, 31 Jul 2019 at 10:22, Carsten Ziegeler <cz...@apache.org> wrote:
> Hi,
>
>
> Georg Henzler wrote
> >>
> >> I assume you're concerned about performance? Without numbers proving
> >> this decreades performance significantly, I wouldn't do any upfront
> >> optimization, especially not if it creates configuration burden.
> >
> >
> > So yes, performance is one part (but most likely not a real problem),
> > the other aspect that for existing projects, there might be false
> > replacements made (I know finding $[] occurrences in existing config
> > values is not that likely, but if for some reason a project has those in
> > their configs it will cause them a major headache to find out what is
> > going on. One good way to mitigate that risk would be to to just leave a
> > reference $[env:key] in the value string unchanged for the case it
> > cannot be replaced (if I read [1] correctly it would currently result in
> > an empty string).
> >
> Ok, I see. Yes, makes sense. I think we should leave the reference in if
> no value is found and probably provide an exclude (regexp) option. This
> way, if you run into the trouble you mention above, you can configure
> the exclude option. But in general you dont have to
>
> Regards
> Carsten
>
> > -Georg
> >
> > [1]
> >
> https://github.com/apache/felix/blob/4c91624c6f1a47c5d6887446cb02fb906b110d40/configadmin-plugins/interpolation/src/main/java/org/apache/felix/configadmin/plugin/interpolation/InterpolationConfigurationPlugin.java#L145
> >
> --
> Carsten Ziegeler
> Adobe Research Switzerland
> cziegeler@apache.org
>
Re: Potential Felix contribution: ConfigAdmin plugin that can
substitute variable placeholders (e.g. for K8s secrets)
Posted by Carsten Ziegeler <cz...@apache.org>.
Hi,
Georg Henzler wrote
>>
>> I assume you're concerned about performance? Without numbers proving
>> this decreades performance significantly, I wouldn't do any upfront
>> optimization, especially not if it creates configuration burden.
>
>
> So yes, performance is one part (but most likely not a real problem),
> the other aspect that for existing projects, there might be false
> replacements made (I know finding $[] occurrences in existing config
> values is not that likely, but if for some reason a project has those in
> their configs it will cause them a major headache to find out what is
> going on. One good way to mitigate that risk would be to to just leave a
> reference $[env:key] in the value string unchanged for the case it
> cannot be replaced (if I read [1] correctly it would currently result in
> an empty string).
>
Ok, I see. Yes, makes sense. I think we should leave the reference in if
no value is found and probably provide an exclude (regexp) option. This
way, if you run into the trouble you mention above, you can configure
the exclude option. But in general you dont have to
Regards
Carsten
> -Georg
>
> [1]
> https://github.com/apache/felix/blob/4c91624c6f1a47c5d6887446cb02fb906b110d40/configadmin-plugins/interpolation/src/main/java/org/apache/felix/configadmin/plugin/interpolation/InterpolationConfigurationPlugin.java#L145
>
--
Carsten Ziegeler
Adobe Research Switzerland
cziegeler@apache.org
Re: Potential Felix contribution: ConfigAdmin plugin that can
substitute variable placeholders (e.g. for K8s secrets)
Posted by Georg Henzler <fe...@ghenzler.de>.
>
> While it might be true that only k8s uses this way atm, still the
> implementation is independent and simply file system based. As this
> configuration prop is tied to the "secret" keyword as a replacement, I
> think renaming it to "secretsdir" makes sense. We might have
> "secretspropertyfile" as you say later on.
>
So if everybody prefers 'secretsdir', that's fine for me (I just think
the format is effectively k8s, we might as well name the property as
such)
> I don't think that's maintainable and creates an unnecessary
> configuration overhead.
> Each time you introduce usage of an interpolation in a configuration
> you need to adjust
> this central configuration as well.
Well if it is a regex, you you can make sure you put the PID in the
"interpolated namespace", so not that much overhead (but yes, more it's
complicated and extra knowledge is required)
>
> I assume you're concerned about performance? Without numbers proving
> this decreades performance significantly, I wouldn't do any upfront
> optimization, especially not if it creates configuration burden.
So yes, performance is one part (but most likely not a real problem),
the other aspect that for existing projects, there might be false
replacements made (I know finding $[] occurrences in existing config
values is not that likely, but if for some reason a project has those in
their configs it will cause them a major headache to find out what is
going on. One good way to mitigate that risk would be to to just leave a
reference $[env:key] in the value string unchanged for the case it
cannot be replaced (if I read [1] correctly it would currently result in
an empty string).
-Georg
[1]
https://github.com/apache/felix/blob/4c91624c6f1a47c5d6887446cb02fb906b110d40/configadmin-plugins/interpolation/src/main/java/org/apache/felix/configadmin/plugin/interpolation/InterpolationConfigurationPlugin.java#L145
Re: Potential Felix contribution: ConfigAdmin plugin that can
substitute variable placeholders (e.g. for K8s secrets)
Posted by Carsten Ziegeler <cz...@apache.org>.
Georg Henzler wrote> Hi David,
>>
>> The $[secret:name] lookup doesn't actually do anything that is k8s
>> specific
>> in itself. K8s just puts the secrets in plain files: the name of the file
>> is the key and the entire content of the file is the secret value. That
>> pattern could also be used in a non-k8s context. Maybe we can rename the
>> property name to
>> 'org.apache.felix.configadmin.plugin.interpolation.secretdir' or
>> something
>> like that?
>
> So even though the secrets directory doesn't contain any k8s-specifc
> tokens, its format is still k8s-specific (I'm not aware of any other
> configuration mechanism that uses a directory with flat files per
> property... there are property files and some languages use yaml files,
> but this mechanisms I have not seen elsewhere). So I think
> org.apache.felix.configadmin.plugin.interpolation.k8sSecretsDir is the
> correct way of specifying this (maybe we get
> org.apache.felix.configadmin.plugin.interpolation.secretsPropertyFile
> and others some time in the future)
>
While it might be true that only k8s uses this way atm, still the
implementation is independent and simply file system based. As this
configuration prop is tied to the "secret" keyword as a replacement, I
think renaming it to "secretsdir" makes sense. We might have
"secretspropertyfile" as you say later on.
>
> And there is one more question: Do we really want to filter all property
> sets for all PIDs? I suppose in reality often only < 5% of the
> configuration data would actually contain placeholders. Would it be
> worthwhile to introduce a property
> org.apache.felix.configadmin.plugin.interpolation.pidRegex to limit the
> PIDs that are transformed?
>
I don't think that's maintainable and creates an unnecessary
configuration overhead.
Each time you introduce usage of an interpolation in a configuration you
need to adjust
this central configuration as well.
I assume you're concerned about performance? Without numbers proving
this decreades performance significantly, I wouldn't do any upfront
optimization, especially not if it creates configuration burden.
Regards
Carsten
--
Carsten Ziegeler
Adobe Research Switzerland
cziegeler@apache.org
Re: Potential Felix contribution: ConfigAdmin plugin that can
substitute variable placeholders (e.g. for K8s secrets)
Posted by Georg Henzler <gh...@apache.org>.
Hi David,
> I guess it would be an idea to make these configurable, however I'm not
> yet
> fully convinced that it would really add much value. Is there a
> specific
> thing you can't do or do you just want to make it look different?
Let's not make it configurable, but I think it is worthwhile to not just
use "a syntax" but rather the best syntax for good reasons :) So I'm
fine with $[] (it's good that it is not just ${} which might collide
with other mechanisms), but I think e.g. #{} would better because
visually, $[] and ${} look very similar, but ${} are easily #{}
distinguished. Whatever we chose now, we will never be able to change
again....
>
> Right yes, it should be possible to escape. Best thing would be to file
> a
> bug for this. There is now a JIRA component for the plugin:
> https://issues.apache.org/jira/projects/FELIX/versions/12345963
I created https://issues.apache.org/jira/browse/FELIX-6164 for this
>
> The $[secret:name] lookup doesn't actually do anything that is k8s
> specific
> in itself. K8s just puts the secrets in plain files: the name of the
> file
> is the key and the entire content of the file is the secret value. That
> pattern could also be used in a non-k8s context. Maybe we can rename
> the
> property name to
> 'org.apache.felix.configadmin.plugin.interpolation.secretdir' or
> something
> like that?
So even though the secrets directory doesn't contain any k8s-specifc
tokens, its format is still k8s-specific (I'm not aware of any other
configuration mechanism that uses a directory with flat files per
property... there are property files and some languages use yaml files,
but this mechanisms I have not seen elsewhere). So I think
org.apache.felix.configadmin.plugin.interpolation.k8sSecretsDir is the
correct way of specifying this (maybe we get
org.apache.felix.configadmin.plugin.interpolation.secretsPropertyFile
and others some time in the future)
And there is one more question: Do we really want to filter all property
sets for all PIDs? I suppose in reality often only < 5% of the
configuration data would actually contain placeholders. Would it be
worthwhile to introduce a property
org.apache.felix.configadmin.plugin.interpolation.pidRegex to limit the
PIDs that are transformed?
-Georg
Re: Potential Felix contribution: ConfigAdmin plugin that can
substitute variable placeholders (e.g. for K8s secrets)
Posted by David Bosschaert <da...@gmail.com>.
Hi Georg,
Thanks for the feedback!
On Mon, 29 Jul 2019 at 19:10, Georg Henzler <gh...@apache.org> wrote:
> Hi David,
>
> great to see this moving forward! Here are a few comments from my side:
>
> * I'm not sure if $[env:name] is the best syntax... I suppose you chose
> that over curly braces to not clash with potential other replacement
> mechanisms downstream. But $[] looks fairly similar to ${}, maybe
> something like %env:name%, %{env:name} or #{env:name} would make it
> visually clearer that it is not a ${} replacement.
>
I guess it would be an idea to make these configurable, however I'm not yet
fully convinced that it would really add much value. Is there a specific
thing you can't do or do you just want to make it look different?
> * It looks like there is no way to escape for the case $[env:name]
> should be treated as literal at the moment - e.g. something like
> \$[env:name] should be possible IMHO
>
Right yes, it should be possible to escape. Best thing would be to file a
bug for this. There is now a JIRA component for the plugin:
https://issues.apache.org/jira/projects/FELIX/versions/12345963
> * I think it's good to use the 'secret' prefix ala $[secret:name]
> generically, but we should also (at least potentially) allow for secret
> stores other than Kubernetes (as described in [1]). I think changing the
> property name org.apache.felix.configadmin.plugin.interpolation.dir [2]
> to org.apache.felix.configadmin.plugin.interpolation.k8sSecretsDir (or
> similar) would be good to clearly indicate the expected directory
> structure as referenced by this property.
>
The $[secret:name] lookup doesn't actually do anything that is k8s specific
in itself. K8s just puts the secrets in plain files: the name of the file
is the key and the entire content of the file is the secret value. That
pattern could also be used in a non-k8s context. Maybe we can rename the
property name to
'org.apache.felix.configadmin.plugin.interpolation.secretdir' or something
like that?
OTOH the plugin could definitely be extended to support other types of
secrets or config values coming out of files. I guess it would be good to
create JIRAs for these if you see the need.
Kind regards,
David
Re: Potential Felix contribution: ConfigAdmin plugin that can
substitute variable placeholders (e.g. for K8s secrets)
Posted by Georg Henzler <gh...@apache.org>.
Hi David,
great to see this moving forward! Here are a few comments from my side:
* I'm not sure if $[env:name] is the best syntax... I suppose you chose
that over curly braces to not clash with potential other replacement
mechanisms downstream. But $[] looks fairly similar to ${}, maybe
something like %env:name%, %{env:name} or #{env:name} would make it
visually clearer that it is not a ${} replacement.
* It looks like there is no way to escape for the case $[env:name]
should be treated as literal at the moment - e.g. something like
\$[env:name] should be possible IMHO
* I think it's good to use the 'secret' prefix ala $[secret:name]
generically, but we should also (at least potentially) allow for secret
stores other than Kubernetes (as described in [1]). I think changing the
property name org.apache.felix.configadmin.plugin.interpolation.dir [2]
to org.apache.felix.configadmin.plugin.interpolation.k8sSecretsDir (or
similar) would be good to clearly indicate the expected directory
structure as referenced by this property.
- Georg
[1]
https://kubernetes.io/docs/concepts/configuration/secret/#using-secrets-as-files-from-a-pod
[2]
https://github.com/apache/felix/blob/4c91624c6f1a47c5d6887446cb02fb906b110d40/configadmin-plugins/interpolation/src/main/java/org/apache/felix/configadmin/plugin/interpolation/Activator.java#L32
On 2019-07-26 13:12, David Bosschaert wrote:
> Thanks for the suggestion, Bertrand!
>
> I have renamed the plugin to 'interpolation'.
> I have also implemented framework/system property based substitution as
> suggested by JB Onofré
> And I have added substitution support for environment variables, which
> can
> be useful when you'd like to configure your runtime (e.g. containers)
> 12-factor style
>
> The code is here:
> https://svn.apache.org/repos/asf/felix/trunk/configadmin-plugins/interpolation
> Documentation is best read on github:
> https://github.com/apache/felix/blob/trunk/configadmin-plugins/interpolation/README.md
>
> I'd like to do an initial 0.0.2 release early next week to make it easy
> for
> everyone to try it out.
>
> Any suggestions, let me know!
>
> Cheers,
>
> David
>
>
> On Fri, 26 Jul 2019 at 10:06, Bertrand Delacretaz
> <bd...@apache.org>
> wrote:
>
>> Hi,
>>
>> On Thu, Jul 25, 2019 at 4:32 PM David Bosschaert
>> <da...@gmail.com> wrote:
>> > ...I was thinking of putting it at configadmin-plugins/substitution..
>>
>> FWIW, "interpolation" is a common term for that as per
>> https://en.wikipedia.org/wiki/String_interpolation
>>
>> -Bertrand
>>
Re: Potential Felix contribution: ConfigAdmin plugin that can
substitute variable placeholders (e.g. for K8s secrets)
Posted by David Bosschaert <da...@gmail.com>.
Thanks for the suggestion, Bertrand!
I have renamed the plugin to 'interpolation'.
I have also implemented framework/system property based substitution as
suggested by JB Onofré
And I have added substitution support for environment variables, which can
be useful when you'd like to configure your runtime (e.g. containers)
12-factor style
The code is here:
https://svn.apache.org/repos/asf/felix/trunk/configadmin-plugins/interpolation
Documentation is best read on github:
https://github.com/apache/felix/blob/trunk/configadmin-plugins/interpolation/README.md
I'd like to do an initial 0.0.2 release early next week to make it easy for
everyone to try it out.
Any suggestions, let me know!
Cheers,
David
On Fri, 26 Jul 2019 at 10:06, Bertrand Delacretaz <bd...@apache.org>
wrote:
> Hi,
>
> On Thu, Jul 25, 2019 at 4:32 PM David Bosschaert
> <da...@gmail.com> wrote:
> > ...I was thinking of putting it at configadmin-plugins/substitution..
>
> FWIW, "interpolation" is a common term for that as per
> https://en.wikipedia.org/wiki/String_interpolation
>
> -Bertrand
>
Re: Potential Felix contribution: ConfigAdmin plugin that can
substitute variable placeholders (e.g. for K8s secrets)
Posted by Bertrand Delacretaz <bd...@apache.org>.
Hi,
On Thu, Jul 25, 2019 at 4:32 PM David Bosschaert
<da...@gmail.com> wrote:
> ...I was thinking of putting it at configadmin-plugins/substitution..
FWIW, "interpolation" is a common term for that as per
https://en.wikipedia.org/wiki/String_interpolation
-Bertrand