You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sling.apache.org by Georg Henzler <sl...@ghenzler.de> on 2016/08/01 06:20:34 UTC
Re: [Context Aware Configs] Merging?
> ... if you have merging (or inheritance) then you change the
> attribute of a parent and this influences all childs and you
> have no idea what happens there...
We can create a felix console plugin that shows for a given
context path and config name the value of the property and the
path from where it was taken - that way both developers and ops
can quickly check what's going wrong if results are unexpected.
> I personally think this comes all down to tooling ...
So this could mean e.g. an additional maven plugin, which handles
merging of configurations and then the runtime can work without
merging. The problem I see with that is that
* you end up having two similar configuration model types (one in
the source code that supports merging and one effective one for
the runtime) - this makes the mechanism harder to understand
for everyone
* the tooling has to be created and IMHO it'll not be easier/less
code than creating a smarter runtime (even if we take into
account that we have to create the felix console plugin to make
it traceable)
Regards
Georg
Re: [Context Aware Configs] Merging?
Posted by Georg Henzler <ge...@netcentric.biz>.
>> * Puppet (not too sure about the others) is not exactly great at dealing
>> with xml files
>> * FileVault [1] is only available in Java, if you want to create proper
>> packages with it you have to call out to java from other languages
>
> Packages are zip files with a defined structure and some required files.
> If you have a template, a few lines of bash (cp -r, sed, zip) will create
> a package containing OSGi configs. This works well in Kubernetes within
> Docker images. (btw, without a package and its version number, the
> OSGIPackageInstaller doesnt install replacement config files)
Simple property filtering works fine yes, but to implement a
replacement for server side config merging you quickly come to a
situation where you would have to read the structure of xml files and
add config xml files and filter rules for it dynamically (think of
creating a config /conf/tenant/myprof/path/to/my/page/myconfig.xml iff
the prop puppetproproot.myprof.path.to.my.page.myconfig.theOneConfigPropToBeOverWrittenHere
exists), doing so using FileVault/XmlApis is more robust/easier than
using manual, text-based assembly.
-Georg
Re: [Context Aware Configs] Merging?
Posted by Ian Boston <ie...@tfd.co.uk>.
Hi,
On 2 August 2016 at 10:30, Georg Henzler <sl...@ghenzler.de> wrote:
> On 2016-08-01 13:29, Konrad Windszus wrote:
>
>> Can't we just use the resource merger approach for merging configs
>> (
>> http://sling.staging.apache.org/documentation/bundles/resource-merger.html
>> <
>> http://sling.staging.apache.org/documentation/bundles/resource-merger.html
>> >)?
>> There probably needs to be a dedicated resource picker for
>> configurations, because neither the search path approach nor the
>> resource super type approach is applicable for configurations.
>>
>
> Sounds like a good match, I could imagine that way we could implement
> merged configs fairly easily!
>
> Clearly with devops automation you have some tooling already in place
>>> which is using text files and does something to start AEM, so you can
>>> add the stuff there - or - the configuration are content packages after
>>> all, so if the automation supports content packages in some form, you're
>>> done.
>>>
>>>
>> Agreed.
>>
>
> DevOps automation is nice, but for generating CRX packages with xml config
> files it's not a great match:
> * Puppet (not too sure about the others) is not exactly great at dealing
> with xml files
> * FileVault [1] is only available in Java, if you want to create proper
> packages with it you have to call out to java from other languages
>
Packages are zip files with a defined structure and some required files.
If you have a template, a few lines of bash (cp -r, sed, zip) will create
a package containing OSGi configs. This works well in Kubernetes within
Docker images. (btw, without a package and its version number, the
OSGIPackageInstaller doesnt install replacement config files)
> * Not having a standard approach makes every project implement their
> homegrown solution (and waste budget where it could really be standard).
> I've have seen a few awkward solutions for generating OSGi configs
> * Alternatively (and this is what I've seen happening the most for OSGi
> configs), people just accept the fact that they have to duplicate
> configuration sets and then run into all the frustration Justin described
> earlier
>
> So I'm still in big favour of having a runtime solution.
>
+1.
Will OSGi configs be context aware or do they only make sense in a Global
context (overridden by /libs /apps )?
I can see how a Factory might generate multiple configs, but OSGi
components are shared and survive outside most Request or Job contexts.
Only where a component is actually created and dies on a thread with
context will context aware OSGi configurations make sense... or am I
missing something ?
Best Regards
Ian
>
> Best Regards
> Georg
>
> [1]
> http://jackrabbit.apache.org/filevault/
>
Re: [Context Aware Configs] Merging?
Posted by Georg Henzler <sl...@ghenzler.de>.
On 2016-08-01 13:29, Konrad Windszus wrote:
> Can't we just use the resource merger approach for merging configs
> (http://sling.staging.apache.org/documentation/bundles/resource-merger.html
> <http://sling.staging.apache.org/documentation/bundles/resource-merger.html>)?
> There probably needs to be a dedicated resource picker for
> configurations, because neither the search path approach nor the
> resource super type approach is applicable for configurations.
Sounds like a good match, I could imagine that way we could implement
merged configs fairly easily!
>> Clearly with devops automation you have some tooling already in place
>> which is using text files and does something to start AEM, so you can
>> add the stuff there - or - the configuration are content packages
>> after
>> all, so if the automation supports content packages in some form,
>> you're
>> done.
>>
>
> Agreed.
DevOps automation is nice, but for generating CRX packages with xml
config files it's not a great match:
* Puppet (not too sure about the others) is not exactly great at dealing
with xml files
* FileVault [1] is only available in Java, if you want to create proper
packages with it you have to call out to java from other languages
* Not having a standard approach makes every project implement their
homegrown solution (and waste budget where it could really be standard).
I've have seen a few awkward solutions for generating OSGi configs
* Alternatively (and this is what I've seen happening the most for OSGi
configs), people just accept the fact that they have to duplicate
configuration sets and then run into all the frustration Justin
described earlier
So I'm still in big favour of having a runtime solution.
Best Regards
Georg
[1]
http://jackrabbit.apache.org/filevault/
Re: [Context Aware Configs] Merging?
Posted by Konrad Windszus <ko...@gmx.de>.
Can't we just use the resource merger approach for merging configs (http://sling.staging.apache.org/documentation/bundles/resource-merger.html <http://sling.staging.apache.org/documentation/bundles/resource-merger.html>)?
There probably needs to be a dedicated resource picker for configurations, because neither the search path approach nor the resource super type approach is applicable for configurations.
That way the merging aspect is nicely separated.
Konrad
> On 01 Aug 2016, at 13:24, Robert Munteanu <ro...@apache.org> wrote:
>
> On Mon, 2016-08-01 at 11:34 +0100, Ian Boston wrote:
>> Hi,
>>
>> On 1 August 2016 at 11:02, Carsten Ziegeler <cz...@apache.org>
>> wrote:
>>
>>>
>>> Well, I think you're misinterpreting my comments - I never said
>>> that it
>>> will require a special tooling - I said it would be insane to edit
>>> a
>>> complex setup by editing nodes in the repository.
>>>
>>
>> I was, apologies and thank you for the clarification.
>>
>> No special tooling *and* a simple set of rules that are natural to
>> communicate and document.
>>
>> I don't have a strong opinion on merging or inheritance, provided its
>> simple to use and the result if obvious with no strange side effects
>> to
>> catch a deployer out.
>
> I think it's possible to have merging on a less fine-grained level, and
> thus more controllable, by using multiple nodes for the same
> configuration instance.
>
> For example:
>
> Global: {
> 'services' : {
> 'serviceA' {
> 'endpoint' : 'https://api.com/foo' <https://api.com/foo'>
> 'params': {
> 'timeout': 500
> }
> }
> }
> }
>
> Site: {
> 'services' : {
> 'serviceA' {
> 'endpoint' : '
> https://stage.api.com/foo' <https://stage.api.com/foo'>
> }
> }
> }
>
> the override only affects the 'endpoint' parameter. If at somepoint we decide to add a 'retryCount' parameter to 'params' this will also be propagated to inherited configurations.
>
> Of course, this comes with the caveats that:
>
> - we need to decide upfront what about how configurations are split into nodes
> - we won't always be able to split configurations into nodes so that all properties are overrideable
>
> Robert
>
>>
>>
>>>
>>>
>>> Clearly with devops automation you have some tooling already in
>>> place
>>> which is using text files and does something to start AEM, so you
>>> can
>>> add the stuff there - or - the configuration are content packages
>>> after
>>> all, so if the automation supports content packages in some form,
>>> you're
>>> done.
>>>
>>
>> Agreed.
>> Best Regards
>> Ian
>>
>>
>>>
>>>
>>> Carsten
>>>
>>>>
>>>> Hi,
>>>>
>>>> I think it must be possible to configure a Sling/AEM instance
>>>> without
>>> using
>>>>
>>>> specific Sling/AEM tooling to create and manage the source
>>>> configuration.
>>>> The reason I say this is almost all serious deployments use some
>>>> form of
>>>> DevOps automation ranging from Puppet/Chef/AWS etc through to
>>>> Docker/Mesos/Kubernetes etc. Almost without exception those
>>>> systems work
>>>> with configuration stored in text files (property, json, yaml
>>>> etc), often
>>>> stored in Git/SVN/Hg, edited by a human in a text editor. There
>>>> are
>>> systems
>>>>
>>>> that only allow config via a GUI, but most of those only used,
>>>> under
>>>> sufferance, when there is no other option, (or competing
>>>> product). The
>>>> "customer" here is not a Sling/AEM expert, but a DevOps engineer
>>>> who has
>>>> never seen either before.
>>>>
>>>> If the only way to configure Sling/AEM is via special Sling/AEM
>>>> tooling
>>>> then we will have made the job of the TechOps/DevOps team even
>>>> harder.
>>>>
>>>> I can think of lots of ways that merging at the property level
>>>> could be
>>>> made to work, but if we are not able to communicate clearly how
>>>> it works,
>>>> and all the potential side effects, to those whose job it is to
>>>> make
>>>> Sling/AEM work in production then we will have failed. If we have
>>>> to
>>> resort
>>>>
>>>> to validation tools, compilers, special editors thats an
>>>> indication the
>>>> system is so complex we are not able to document it in plain
>>>> english.
>>> For
>>>>
>>>> that reason, I beg, please keep whatever solution is chosen as
>>>> simple as
>>>> possible.
>>>>
>>>> I did try to document "object" level inheritance (ie aggregating
>>>> properties), with all the side effects, but when I realised it
>>>> was not
>>>> going to be simple, I gave up and went back "object" level
>>>> merging, which
>>>> is brutal and simple by comparison.
>>>>
>>>> Best Regards
>>>> Ian
>>>>
>>>>
>>>>
>>>> On 1 August 2016 at 08:29, Carsten Ziegeler <cziegeler@apache.org
>>>>>
>>> wrote:
>>>>
>>>>
>>>>>
>>>>>>
>>>>>> Hi everyone,
>>>>>>
>>>>>> I personally see a tooling based approach complicated due to
>>>>>> the
>>>>> ownership
>>>>>>
>>>>>> scenarios.
>>>>> Let's assume two scenarios, configuration A and configuration B
>>>>> inherits
>>>>> from A.
>>>>>
>>>>> Now, if we support merging, someone is allowed to change A,
>>>>> adds a new
>>>>> property and this automatically influences B - and in this case
>>>>> the
>>>>> person who made changes to A does not even need to have access
>>>>> rights to
>>>>> change anything for B. (Which I think is scary).
>>>>>
>>>>> If we don't support inheritance, there is still someone who is
>>>>> allowed
>>>>> to make a change to A - at first this has no influence on B. If
>>>>> that
>>>>> same person is allowed to also change B and wants to "inherit"
>>>>> the new
>>>>> property to B, the tooling will allow so.
>>>>>
>>>>> With that respect, not having automatic inheritance sounds like
>>>>> the
>>>>> better option when it comes to access rights.
>>>>>
>>>>> Carsten
>>>>>
>>>>>>
>>>>>> We are talking about context aware configuration that
>>>>>> typically
>>>>>> would be applied by some runtime user that might have limited
>>>>>> write
>>>>> access
>>>>>>
>>>>>> (just for a specific context). The assumption that who writes
>>>>>> to the
>>>>>> inheriting configuration node also has write access to the
>>>>>> subsequent
>>>>>> levels is something we cannot and should not make here. I
>>>>>> know multiple
>>>>>> customer scenarios where a central site (including global
>>> configuration)
>>>>
>>>>>
>>>>>>
>>>>>> was maintained by a central marketing organization while the
>>>>>> markets
>>>>> owned
>>>>>>
>>>>>> their own sites - so they declare inheritance (pull) instead
>>>>>> of having
>>>>> the
>>>>>>
>>>>>> global one pushed.
>>>>>>
>>>>>> The one thing in between that I could imagine might work
>>>>>> sufficiently
>>> is
>>>>
>>>>>
>>>>>>
>>>>>> referencing (by path to property) instead of inheritance.
>>>>>> This still
>>>>>> provides the lookup of "inheritance" at runtime where changes
>>>>>> are
>>>>> reflected
>>>>>>
>>>>>> in the lookup immediately but at the same time gets rid of
>>>>>> magic
>>>>>> inheritance rules.
>>>>>>
>>>>>> Just a thought about an alternative in between.
>>>>>>
>>>>>> Cheers
>>>>>> Dominik
>>>>>>
>>>>>> On Mon, Aug 1, 2016 at 8:36 AM, Carsten Ziegeler <cziegeler@a
>>>>>> pache.org
>>>>
>>>>>
>>>>>>
>>>>>> wrote:
>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> ... if you have merging (or inheritance) then you
>>>>>>>>> change the
>>>>>>>>> attribute of a parent and this influences all childs
>>>>>>>>> and you
>>>>>>>>> have no idea what happens there...
>>>>>>>>
>>>>>>>> We can create a felix console plugin that shows for a
>>>>>>>> given
>>>>>>>> context path and config name the value of the property
>>>>>>>> and the
>>>>>>>> path from where it was taken - that way both developers
>>>>>>>> and ops
>>>>>>>> can quickly check what's going wrong if results are
>>>>>>>> unexpected.
>>>>>>>>
>>>>>>>>>
>>>>>>>>> I personally think this comes all down to tooling ...
>>>>>>>>
>>>>>>>> So this could mean e.g. an additional maven plugin, which
>>>>>>>> handles
>>>>>>>> merging of configurations and then the runtime can work
>>>>>>>> without
>>>>>>>> merging. The problem I see with that is that
>>>>>>>> * you end up having two similar configuration model types
>>>>>>>> (one in
>>>>>>>> the source code that supports merging and one effective
>>>>>>>> one for
>>>>>>>> the runtime) - this makes the mechanism harder to
>>>>>>>> understand
>>>>>>>> for everyone
>>>>>>>> * the tooling has to be created and IMHO it'll not be
>>>>>>>> easier/less
>>>>>>>> code than creating a smarter runtime (even if we take
>>>>>>>> into
>>>>>>>> account that we have to create the felix console plugin
>>>>>>>> to make
>>>>>>>> it traceable)
>>>>>>>>
>>>>>>> First of all, you need tooling anyway. It would be foolish
>>>>>>> to make
>>>>>>> these configurations without any tooling by hand. Tooling
>>>>>>> can easily
>>>>>>> be customized to anyones needs without tampering the
>>>>>>> implementation.
>>>>>>> The implementation at runtime can be simply and effective.
>>>>>>>
>>>>>>> We had similar discussions while starting the work for the
>>>>>>> OSGi
>>>>>>> Configurator which allows you to put OSGi configurations
>>>>>>> into a bundle
>>>>>>> which then get applied at runtime. My initial proposal had
>>>>>>> all the
>>>>>>> things requested here as well (merging, inheritance etc.).
>>>>>>> It turned
>>>>>>> out that this messed up the specification and made the
>>>>>>> implementation
>>>>>>> extraordinary complicated. While moving this into the
>>>>>>> tooling which
>>>>>>> creates the bundles in the first place solves all the
>>>>>>> problems.
>>>>>>>
>>>>>>> Just think of all the different cases you have to solve
>>>>>>> once you
>>>>>>> start support merging. And not to mention the runtime
>>>>>>> implications as
>>>>>>> you have to do the merging dynamically at runtime as well,
>>>>>>> or you
>>>>>>> start introducing all kinds of caching again. All of this
>>>>>>> is not
>>>>>>> required if you move this to the tooling level.
>>>>>>>
>>>>>>> Regards
>>>>>>>
>>>>>>> Carsten
>>>>>>>
>>>>>>> --
>>>>>>> Carsten Ziegeler
>>>>>>> Adobe Research Switzerland
>>>>>>> cziegeler@apache.org
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Carsten Ziegeler
>>>>> Adobe Research Switzerland
>>>>> cziegeler@apache.org
>>>>>
>>>>>
>>>>
>>>
>>>
>>>
>>>
>>> --
>>> Carsten Ziegeler
>>> Adobe Research Switzerland
>>> cziegeler@apache.org
Re: [Context Aware Configs] Merging?
Posted by Justin Edelson <ju...@justinedelson.com>.
On Mon, Aug 1, 2016 at 7:24 AM Robert Munteanu <ro...@apache.org> wrote:
> On Mon, 2016-08-01 at 11:34 +0100, Ian Boston wrote:
> > Hi,
> >
> > On 1 August 2016 at 11:02, Carsten Ziegeler <cz...@apache.org>
> > wrote:
> >
> > >
> > > Well, I think you're misinterpreting my comments - I never said
> > > that it
> > > will require a special tooling - I said it would be insane to edit
> > > a
> > > complex setup by editing nodes in the repository.
> > >
> >
> > I was, apologies and thank you for the clarification.
> >
> > No special tooling *and* a simple set of rules that are natural to
> > communicate and document.
> >
> > I don't have a strong opinion on merging or inheritance, provided its
> > simple to use and the result if obvious with no strange side effects
> > to
> > catch a deployer out.
>
> I think it's possible to have merging on a less fine-grained level, and
> thus more controllable, by using multiple nodes for the same
> configuration instance.
>
> For example:
>
> Global: {
> 'services' : {
> 'serviceA' {
> 'endpoint' : 'https://api.com/foo'
> 'params': {
> 'timeout': 500
> }
> }
> }
> }
>
> Site: {
> 'services' : {
> 'serviceA' {
> 'endpoint' : '
> https://stage.api.com/foo'
> }
> }
> }
>
> the override only affects the 'endpoint' parameter. If at somepoint we
> decide to add a 'retryCount' parameter to 'params' this will also be
> propagated to inherited configurations.
>
I feel like this example only makes the argument for property-level merging
all the more obvious. If we support only object-level merging, then you
will inevitably end up with a state like this:
Global: {
'services' : {
'serviceA' {
'endpoint' : 'https://api.com/foo',
'params': {
'timeout': 500
},
'params2' : {
'connectTimeout' : 5000
}
}
}
}
Site: {
'services' : {
'serviceA' {
'endpoint' : 'https://stage.api.com/foo',
'params': {
'timeout': 200
}
}
}
}
where the fact that the 'params' object has been overridden forces the
global config editor to define a 'params2' object to store some new
properties which will be inherited and then when those are overridden, a
'params3' and so on.
Justin
>
> Of course, this comes with the caveats that:
>
> - we need to decide upfront what about how configurations are split into
> nodes
> - we won't always be able to split configurations into nodes so that all
> properties are overrideable
> Robert
>
> >
> >
> > >
> > >
> > > Clearly with devops automation you have some tooling already in
> > > place
> > > which is using text files and does something to start AEM, so you
> > > can
> > > add the stuff there - or - the configuration are content packages
> > > after
> > > all, so if the automation supports content packages in some form,
> > > you're
> > > done.
> > >
> >
> > Agreed.
> > Best Regards
> > Ian
> >
> >
> > >
> > >
> > > Carsten
> > >
> > > >
> > > > Hi,
> > > >
> > > > I think it must be possible to configure a Sling/AEM instance
> > > > without
> > > using
> > > >
> > > > specific Sling/AEM tooling to create and manage the source
> > > > configuration.
> > > > The reason I say this is almost all serious deployments use some
> > > > form of
> > > > DevOps automation ranging from Puppet/Chef/AWS etc through to
> > > > Docker/Mesos/Kubernetes etc. Almost without exception those
> > > > systems work
> > > > with configuration stored in text files (property, json, yaml
> > > > etc), often
> > > > stored in Git/SVN/Hg, edited by a human in a text editor. There
> > > > are
> > > systems
> > > >
> > > > that only allow config via a GUI, but most of those only used,
> > > > under
> > > > sufferance, when there is no other option, (or competing
> > > > product). The
> > > > "customer" here is not a Sling/AEM expert, but a DevOps engineer
> > > > who has
> > > > never seen either before.
> > > >
> > > > If the only way to configure Sling/AEM is via special Sling/AEM
> > > > tooling
> > > > then we will have made the job of the TechOps/DevOps team even
> > > > harder.
> > > >
> > > > I can think of lots of ways that merging at the property level
> > > > could be
> > > > made to work, but if we are not able to communicate clearly how
> > > > it works,
> > > > and all the potential side effects, to those whose job it is to
> > > > make
> > > > Sling/AEM work in production then we will have failed. If we have
> > > > to
> > > resort
> > > >
> > > > to validation tools, compilers, special editors thats an
> > > > indication the
> > > > system is so complex we are not able to document it in plain
> > > > english.
> > > For
> > > >
> > > > that reason, I beg, please keep whatever solution is chosen as
> > > > simple as
> > > > possible.
> > > >
> > > > I did try to document "object" level inheritance (ie aggregating
> > > > properties), with all the side effects, but when I realised it
> > > > was not
> > > > going to be simple, I gave up and went back "object" level
> > > > merging, which
> > > > is brutal and simple by comparison.
> > > >
> > > > Best Regards
> > > > Ian
> > > >
> > > >
> > > >
> > > > On 1 August 2016 at 08:29, Carsten Ziegeler <cziegeler@apache.org
> > > > >
> > > wrote:
> > > >
> > > >
> > > > >
> > > > > >
> > > > > > Hi everyone,
> > > > > >
> > > > > > I personally see a tooling based approach complicated due to
> > > > > > the
> > > > > ownership
> > > > > >
> > > > > > scenarios.
> > > > > Let's assume two scenarios, configuration A and configuration B
> > > > > inherits
> > > > > from A.
> > > > >
> > > > > Now, if we support merging, someone is allowed to change A,
> > > > > adds a new
> > > > > property and this automatically influences B - and in this case
> > > > > the
> > > > > person who made changes to A does not even need to have access
> > > > > rights to
> > > > > change anything for B. (Which I think is scary).
> > > > >
> > > > > If we don't support inheritance, there is still someone who is
> > > > > allowed
> > > > > to make a change to A - at first this has no influence on B. If
> > > > > that
> > > > > same person is allowed to also change B and wants to "inherit"
> > > > > the new
> > > > > property to B, the tooling will allow so.
> > > > >
> > > > > With that respect, not having automatic inheritance sounds like
> > > > > the
> > > > > better option when it comes to access rights.
> > > > >
> > > > > Carsten
> > > > >
> > > > > >
> > > > > > We are talking about context aware configuration that
> > > > > > typically
> > > > > > would be applied by some runtime user that might have limited
> > > > > > write
> > > > > access
> > > > > >
> > > > > > (just for a specific context). The assumption that who writes
> > > > > > to the
> > > > > > inheriting configuration node also has write access to the
> > > > > > subsequent
> > > > > > levels is something we cannot and should not make here. I
> > > > > > know multiple
> > > > > > customer scenarios where a central site (including global
> > > configuration)
> > > >
> > > > >
> > > > > >
> > > > > > was maintained by a central marketing organization while the
> > > > > > markets
> > > > > owned
> > > > > >
> > > > > > their own sites - so they declare inheritance (pull) instead
> > > > > > of having
> > > > > the
> > > > > >
> > > > > > global one pushed.
> > > > > >
> > > > > > The one thing in between that I could imagine might work
> > > > > > sufficiently
> > > is
> > > >
> > > > >
> > > > > >
> > > > > > referencing (by path to property) instead of inheritance.
> > > > > > This still
> > > > > > provides the lookup of "inheritance" at runtime where changes
> > > > > > are
> > > > > reflected
> > > > > >
> > > > > > in the lookup immediately but at the same time gets rid of
> > > > > > magic
> > > > > > inheritance rules.
> > > > > >
> > > > > > Just a thought about an alternative in between.
> > > > > >
> > > > > > Cheers
> > > > > > Dominik
> > > > > >
> > > > > > On Mon, Aug 1, 2016 at 8:36 AM, Carsten Ziegeler <cziegeler@a
> > > > > > pache.org
> > > >
> > > > >
> > > > > >
> > > > > > wrote:
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > ... if you have merging (or inheritance) then you
> > > > > > > > > change the
> > > > > > > > > attribute of a parent and this influences all childs
> > > > > > > > > and you
> > > > > > > > > have no idea what happens there...
> > > > > > > >
> > > > > > > > We can create a felix console plugin that shows for a
> > > > > > > > given
> > > > > > > > context path and config name the value of the property
> > > > > > > > and the
> > > > > > > > path from where it was taken - that way both developers
> > > > > > > > and ops
> > > > > > > > can quickly check what's going wrong if results are
> > > > > > > > unexpected.
> > > > > > > >
> > > > > > > > >
> > > > > > > > > I personally think this comes all down to tooling ...
> > > > > > > >
> > > > > > > > So this could mean e.g. an additional maven plugin, which
> > > > > > > > handles
> > > > > > > > merging of configurations and then the runtime can work
> > > > > > > > without
> > > > > > > > merging. The problem I see with that is that
> > > > > > > > * you end up having two similar configuration model types
> > > > > > > > (one in
> > > > > > > > the source code that supports merging and one effective
> > > > > > > > one for
> > > > > > > > the runtime) - this makes the mechanism harder to
> > > > > > > > understand
> > > > > > > > for everyone
> > > > > > > > * the tooling has to be created and IMHO it'll not be
> > > > > > > > easier/less
> > > > > > > > code than creating a smarter runtime (even if we take
> > > > > > > > into
> > > > > > > > account that we have to create the felix console plugin
> > > > > > > > to make
> > > > > > > > it traceable)
> > > > > > > >
> > > > > > > First of all, you need tooling anyway. It would be foolish
> > > > > > > to make
> > > > > > > these configurations without any tooling by hand. Tooling
> > > > > > > can easily
> > > > > > > be customized to anyones needs without tampering the
> > > > > > > implementation.
> > > > > > > The implementation at runtime can be simply and effective.
> > > > > > >
> > > > > > > We had similar discussions while starting the work for the
> > > > > > > OSGi
> > > > > > > Configurator which allows you to put OSGi configurations
> > > > > > > into a bundle
> > > > > > > which then get applied at runtime. My initial proposal had
> > > > > > > all the
> > > > > > > things requested here as well (merging, inheritance etc.).
> > > > > > > It turned
> > > > > > > out that this messed up the specification and made the
> > > > > > > implementation
> > > > > > > extraordinary complicated. While moving this into the
> > > > > > > tooling which
> > > > > > > creates the bundles in the first place solves all the
> > > > > > > problems.
> > > > > > >
> > > > > > > Just think of all the different cases you have to solve
> > > > > > > once you
> > > > > > > start support merging. And not to mention the runtime
> > > > > > > implications as
> > > > > > > you have to do the merging dynamically at runtime as well,
> > > > > > > or you
> > > > > > > start introducing all kinds of caching again. All of this
> > > > > > > is not
> > > > > > > required if you move this to the tooling level.
> > > > > > >
> > > > > > > Regards
> > > > > > >
> > > > > > > Carsten
> > > > > > >
> > > > > > > --
> > > > > > > Carsten Ziegeler
> > > > > > > Adobe Research Switzerland
> > > > > > > cziegeler@apache.org
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Carsten Ziegeler
> > > > > Adobe Research Switzerland
> > > > > cziegeler@apache.org
> > > > >
> > > > >
> > > >
> > >
> > >
> > >
> > >
> > > --
> > > Carsten Ziegeler
> > > Adobe Research Switzerland
> > > cziegeler@apache.org
> > >
> > >
>
>
Re: [Context Aware Configs] Merging?
Posted by Robert Munteanu <ro...@apache.org>.
On Mon, 2016-08-01 at 11:34 +0100, Ian Boston wrote:
> Hi,
>
> On 1 August 2016 at 11:02, Carsten Ziegeler <cz...@apache.org>
> wrote:
>
> >
> > Well, I think you're misinterpreting my comments - I never said
> > that it
> > will require a special tooling - I said it would be insane to edit
> > a
> > complex setup by editing nodes in the repository.
> >
>
> I was, apologies and thank you for the clarification.
>
> No special tooling *and* a simple set of rules that are natural to
> communicate and document.
>
> I don't have a strong opinion on merging or inheritance, provided its
> simple to use and the result if obvious with no strange side effects
> to
> catch a deployer out.
I think it's possible to have merging on a less fine-grained level, and
thus more controllable, by using multiple nodes for the same
configuration instance.
For example:
Global: {
����'services' : {
��������'serviceA' {
������������'endpoint' : 'https://api.com/foo'
� � � � � � 'params': {
� � � � � � � �'timeout': 500
� � � � � � }
��������}
����}
}
Site: {
����'services' : {
��������'serviceA' {
������������'endpoint' : '
https://stage.api.com/foo'
��������}
����}
}
the override only affects the 'endpoint' parameter. If at somepoint we decide to add a 'retryCount' parameter to 'params' this will also be propagated to inherited configurations.
Of course, this comes with the caveats that:
- we�need to decide upfront what about how configurations are split into nodes
- we won't always be able to split configurations into nodes so that all properties are overrideable
Robert
>
>
> >
> >
> > Clearly with devops automation you have some tooling already in
> > place
> > which is using text files and does something to start AEM, so you
> > can
> > add the stuff there - or - the configuration are content packages
> > after
> > all, so if the automation supports content packages in some form,
> > you're
> > done.
> >
>
> Agreed.
> Best Regards
> Ian
>
>
> >
> >
> > Carsten
> >
> > >
> > > Hi,
> > >
> > > I think it must be possible to configure a Sling/AEM instance
> > > without
> > using
> > >
> > > specific Sling/AEM tooling to create and manage the source
> > > configuration.
> > > The reason I say this is almost all serious deployments use some
> > > form of
> > > DevOps automation ranging from Puppet/Chef/AWS etc through to
> > > Docker/Mesos/Kubernetes etc. Almost without exception those
> > > systems work
> > > with configuration stored in text files (property, json, yaml
> > > etc), often
> > > stored in Git/SVN/Hg, edited by a human in a text editor. There
> > > are
> > systems
> > >
> > > that only allow config via a GUI, but most of those only used,
> > > under
> > > sufferance, when there is no other option, (or competing
> > > product). The
> > > "customer" here is not a Sling/AEM expert, but a DevOps engineer
> > > who has
> > > never seen either before.
> > >
> > > If the only way to configure Sling/AEM is via special Sling/AEM
> > > tooling
> > > then we will have made the job of the TechOps/DevOps team even
> > > harder.
> > >
> > > I can think of lots of ways that merging at the property level
> > > could be
> > > made to work, but if we are not able to communicate clearly how
> > > it works,
> > > and all the potential side effects, to those whose job it is to
> > > make
> > > Sling/AEM work in production then we will have failed. If we have
> > > to
> > resort
> > >
> > > to validation tools, compilers, special editors thats an
> > > indication the
> > > system is so complex we are not able to document it in plain
> > > english.
> > For
> > >
> > > that reason, I beg, please keep whatever solution is chosen as
> > > simple as
> > > possible.
> > >
> > > I did try to document "object" level inheritance (ie aggregating
> > > properties), with all the side effects, but when I realised it
> > > was not
> > > going to be simple, I gave up and went back "object" level
> > > merging, which
> > > is brutal and simple by comparison.
> > >
> > > Best Regards
> > > Ian
> > >
> > >
> > >
> > > On 1 August 2016 at 08:29, Carsten Ziegeler <cziegeler@apache.org
> > > >
> > wrote:
> > >
> > >
> > > >
> > > > >
> > > > > Hi everyone,
> > > > >
> > > > > I personally see a tooling based approach complicated due to
> > > > > the
> > > > ownership
> > > > >
> > > > > scenarios.
> > > > Let's assume two scenarios, configuration A and configuration B
> > > > inherits
> > > > from A.
> > > >
> > > > Now, if we support merging, someone is allowed to change A,
> > > > adds a new
> > > > property and this automatically influences B - and in this case
> > > > the
> > > > person who made changes to A does not even need to have access
> > > > rights to
> > > > change anything for B. (Which I think is scary).
> > > >
> > > > If we don't support inheritance, there is still someone who is
> > > > allowed
> > > > to make a change to A - at first this has no influence on B. If
> > > > that
> > > > same person is allowed to also change B and wants to "inherit"
> > > > the new
> > > > property to B, the tooling will allow so.
> > > >
> > > > With that respect, not having automatic inheritance sounds like
> > > > the
> > > > better option when it comes to access rights.
> > > >
> > > > Carsten
> > > >
> > > > >
> > > > > We are talking about context aware configuration that
> > > > > typically
> > > > > would be applied by some runtime user that might have limited
> > > > > write
> > > > access
> > > > >
> > > > > (just for a specific context). The assumption that who writes
> > > > > to the
> > > > > inheriting configuration node also has write access to the
> > > > > subsequent
> > > > > levels is something we cannot and should not make here. I
> > > > > know multiple
> > > > > customer scenarios where a central site (including global
> > configuration)
> > >
> > > >
> > > > >
> > > > > was maintained by a central marketing organization while the
> > > > > markets
> > > > owned
> > > > >
> > > > > their own sites - so they declare inheritance (pull) instead
> > > > > of having
> > > > the
> > > > >
> > > > > global one pushed.
> > > > >
> > > > > The one thing in between that I could imagine might work
> > > > > sufficiently
> > is
> > >
> > > >
> > > > >
> > > > > referencing (by path to property) instead of inheritance.
> > > > > This still
> > > > > provides the lookup of "inheritance" at runtime where changes
> > > > > are
> > > > reflected
> > > > >
> > > > > in the lookup immediately but at the same time gets rid of
> > > > > magic
> > > > > inheritance rules.
> > > > >
> > > > > Just a thought about an alternative in between.
> > > > >
> > > > > Cheers
> > > > > Dominik
> > > > >
> > > > > On Mon, Aug 1, 2016 at 8:36 AM, Carsten Ziegeler <cziegeler@a
> > > > > pache.org
> > >
> > > >
> > > > >
> > > > > wrote:
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > ... if you have merging (or inheritance) then you
> > > > > > > > change the
> > > > > > > > attribute of a parent and this influences all childs
> > > > > > > > and you
> > > > > > > > have no idea what happens there...
> > > > > > >
> > > > > > > We can create a felix console plugin that shows for a
> > > > > > > given
> > > > > > > context path and config name the value of the property
> > > > > > > and the
> > > > > > > path from where it was taken - that way both developers
> > > > > > > and ops
> > > > > > > can quickly check what's going wrong if results are
> > > > > > > unexpected.
> > > > > > >
> > > > > > > >
> > > > > > > > I personally think this comes all down to tooling ...
> > > > > > >
> > > > > > > So this could mean e.g. an additional maven plugin, which
> > > > > > > handles
> > > > > > > merging of configurations and then the runtime can work
> > > > > > > without
> > > > > > > merging. The problem I see with that is that
> > > > > > > * you end up having two similar configuration model types
> > > > > > > (one in
> > > > > > > � the source code that supports merging and one effective
> > > > > > > one for
> > > > > > > � the runtime) - this makes the mechanism harder to
> > > > > > > understand
> > > > > > > � for everyone
> > > > > > > * the tooling has to be created and IMHO it'll not be
> > > > > > > easier/less
> > > > > > > � code than creating a smarter runtime (even if we take
> > > > > > > into
> > > > > > > � account that we have to create the felix console plugin
> > > > > > > to make
> > > > > > > � it traceable)
> > > > > > >
> > > > > > First of all, you need tooling anyway. It would be foolish
> > > > > > to make
> > > > > > these configurations without any tooling by hand. Tooling
> > > > > > can easily
> > > > > > be customized to anyones needs without tampering the
> > > > > > implementation.
> > > > > > The implementation at runtime can be simply and effective.
> > > > > >
> > > > > > We had similar discussions while starting the work for the
> > > > > > OSGi
> > > > > > Configurator which allows you to put OSGi configurations
> > > > > > into a bundle
> > > > > > which then get applied at runtime. My initial proposal had
> > > > > > all the
> > > > > > things requested here as well (merging, inheritance etc.).
> > > > > > It turned
> > > > > > out that this messed up the specification and made the
> > > > > > implementation
> > > > > > extraordinary complicated. While moving this into the
> > > > > > tooling which
> > > > > > creates the bundles in the first place solves all the
> > > > > > problems.
> > > > > >
> > > > > > Just think of all the different cases you have to solve
> > > > > > once you
> > > > > > start support merging. And not to mention the runtime
> > > > > > implications as
> > > > > > you have to do the merging dynamically at runtime as well,
> > > > > > or you
> > > > > > start introducing all kinds of caching again. All of this
> > > > > > is not
> > > > > > required if you move this to the tooling level.
> > > > > >
> > > > > > �Regards
> > > > > >
> > > > > > Carsten
> > > > > >
> > > > > > --
> > > > > > Carsten Ziegeler
> > > > > > Adobe Research Switzerland
> > > > > > cziegeler@apache.org
> > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Carsten Ziegeler
> > > > Adobe Research Switzerland
> > > > cziegeler@apache.org
> > > >
> > > >
> > >
> >
> >
> >
> >
> > --
> > Carsten Ziegeler
> > Adobe Research Switzerland
> > cziegeler@apache.org
> >
> >
Re: [Context Aware Configs] Merging?
Posted by Ian Boston <ie...@tfd.co.uk>.
Hi,
On 1 August 2016 at 11:02, Carsten Ziegeler <cz...@apache.org> wrote:
> Well, I think you're misinterpreting my comments - I never said that it
> will require a special tooling - I said it would be insane to edit a
> complex setup by editing nodes in the repository.
>
I was, apologies and thank you for the clarification.
No special tooling *and* a simple set of rules that are natural to
communicate and document.
I don't have a strong opinion on merging or inheritance, provided its
simple to use and the result if obvious with no strange side effects to
catch a deployer out.
>
> Clearly with devops automation you have some tooling already in place
> which is using text files and does something to start AEM, so you can
> add the stuff there - or - the configuration are content packages after
> all, so if the automation supports content packages in some form, you're
> done.
>
Agreed.
Best Regards
Ian
>
> Carsten
>
> > Hi,
> >
> > I think it must be possible to configure a Sling/AEM instance without
> using
> > specific Sling/AEM tooling to create and manage the source configuration.
> > The reason I say this is almost all serious deployments use some form of
> > DevOps automation ranging from Puppet/Chef/AWS etc through to
> > Docker/Mesos/Kubernetes etc. Almost without exception those systems work
> > with configuration stored in text files (property, json, yaml etc), often
> > stored in Git/SVN/Hg, edited by a human in a text editor. There are
> systems
> > that only allow config via a GUI, but most of those only used, under
> > sufferance, when there is no other option, (or competing product). The
> > "customer" here is not a Sling/AEM expert, but a DevOps engineer who has
> > never seen either before.
> >
> > If the only way to configure Sling/AEM is via special Sling/AEM tooling
> > then we will have made the job of the TechOps/DevOps team even harder.
> >
> > I can think of lots of ways that merging at the property level could be
> > made to work, but if we are not able to communicate clearly how it works,
> > and all the potential side effects, to those whose job it is to make
> > Sling/AEM work in production then we will have failed. If we have to
> resort
> > to validation tools, compilers, special editors thats an indication the
> > system is so complex we are not able to document it in plain english.
> For
> > that reason, I beg, please keep whatever solution is chosen as simple as
> > possible.
> >
> > I did try to document "object" level inheritance (ie aggregating
> > properties), with all the side effects, but when I realised it was not
> > going to be simple, I gave up and went back "object" level merging, which
> > is brutal and simple by comparison.
> >
> > Best Regards
> > Ian
> >
> >
> >
> > On 1 August 2016 at 08:29, Carsten Ziegeler <cz...@apache.org>
> wrote:
> >
> >>> Hi everyone,
> >>>
> >>> I personally see a tooling based approach complicated due to the
> >> ownership
> >>> scenarios.
> >> Let's assume two scenarios, configuration A and configuration B inherits
> >> from A.
> >>
> >> Now, if we support merging, someone is allowed to change A, adds a new
> >> property and this automatically influences B - and in this case the
> >> person who made changes to A does not even need to have access rights to
> >> change anything for B. (Which I think is scary).
> >>
> >> If we don't support inheritance, there is still someone who is allowed
> >> to make a change to A - at first this has no influence on B. If that
> >> same person is allowed to also change B and wants to "inherit" the new
> >> property to B, the tooling will allow so.
> >>
> >> With that respect, not having automatic inheritance sounds like the
> >> better option when it comes to access rights.
> >>
> >> Carsten
> >>
> >>> We are talking about context aware configuration that typically
> >>> would be applied by some runtime user that might have limited write
> >> access
> >>> (just for a specific context). The assumption that who writes to the
> >>> inheriting configuration node also has write access to the subsequent
> >>> levels is something we cannot and should not make here. I know multiple
> >>> customer scenarios where a central site (including global
> configuration)
> >>> was maintained by a central marketing organization while the markets
> >> owned
> >>> their own sites - so they declare inheritance (pull) instead of having
> >> the
> >>> global one pushed.
> >>>
> >>> The one thing in between that I could imagine might work sufficiently
> is
> >>> referencing (by path to property) instead of inheritance. This still
> >>> provides the lookup of "inheritance" at runtime where changes are
> >> reflected
> >>> in the lookup immediately but at the same time gets rid of magic
> >>> inheritance rules.
> >>>
> >>> Just a thought about an alternative in between.
> >>>
> >>> Cheers
> >>> Dominik
> >>>
> >>> On Mon, Aug 1, 2016 at 8:36 AM, Carsten Ziegeler <cziegeler@apache.org
> >
> >>> wrote:
> >>>
> >>>>>> ... if you have merging (or inheritance) then you change the
> >>>>>> attribute of a parent and this influences all childs and you
> >>>>>> have no idea what happens there...
> >>>>>
> >>>>> We can create a felix console plugin that shows for a given
> >>>>> context path and config name the value of the property and the
> >>>>> path from where it was taken - that way both developers and ops
> >>>>> can quickly check what's going wrong if results are unexpected.
> >>>>>
> >>>>>> I personally think this comes all down to tooling ...
> >>>>>
> >>>>> So this could mean e.g. an additional maven plugin, which handles
> >>>>> merging of configurations and then the runtime can work without
> >>>>> merging. The problem I see with that is that
> >>>>> * you end up having two similar configuration model types (one in
> >>>>> the source code that supports merging and one effective one for
> >>>>> the runtime) - this makes the mechanism harder to understand
> >>>>> for everyone
> >>>>> * the tooling has to be created and IMHO it'll not be easier/less
> >>>>> code than creating a smarter runtime (even if we take into
> >>>>> account that we have to create the felix console plugin to make
> >>>>> it traceable)
> >>>>>
> >>>> First of all, you need tooling anyway. It would be foolish to make
> >>>> these configurations without any tooling by hand. Tooling can easily
> >>>> be customized to anyones needs without tampering the implementation.
> >>>> The implementation at runtime can be simply and effective.
> >>>>
> >>>> We had similar discussions while starting the work for the OSGi
> >>>> Configurator which allows you to put OSGi configurations into a bundle
> >>>> which then get applied at runtime. My initial proposal had all the
> >>>> things requested here as well (merging, inheritance etc.). It turned
> >>>> out that this messed up the specification and made the implementation
> >>>> extraordinary complicated. While moving this into the tooling which
> >>>> creates the bundles in the first place solves all the problems.
> >>>>
> >>>> Just think of all the different cases you have to solve once you
> >>>> start support merging. And not to mention the runtime implications as
> >>>> you have to do the merging dynamically at runtime as well, or you
> >>>> start introducing all kinds of caching again. All of this is not
> >>>> required if you move this to the tooling level.
> >>>>
> >>>> Regards
> >>>>
> >>>> Carsten
> >>>>
> >>>> --
> >>>> Carsten Ziegeler
> >>>> Adobe Research Switzerland
> >>>> cziegeler@apache.org
> >>>>
> >>>>
> >>>
> >>
> >>
> >>
> >>
> >> --
> >> Carsten Ziegeler
> >> Adobe Research Switzerland
> >> cziegeler@apache.org
> >>
> >>
> >
>
>
>
>
> --
> Carsten Ziegeler
> Adobe Research Switzerland
> cziegeler@apache.org
>
>
Re: [Context Aware Configs] Merging?
Posted by Carsten Ziegeler <cz...@apache.org>.
Well, I think you're misinterpreting my comments - I never said that it
will require a special tooling - I said it would be insane to edit a
complex setup by editing nodes in the repository.
Clearly with devops automation you have some tooling already in place
which is using text files and does something to start AEM, so you can
add the stuff there - or - the configuration are content packages after
all, so if the automation supports content packages in some form, you're
done.
Carsten
> Hi,
>
> I think it must be possible to configure a Sling/AEM instance without using
> specific Sling/AEM tooling to create and manage the source configuration.
> The reason I say this is almost all serious deployments use some form of
> DevOps automation ranging from Puppet/Chef/AWS etc through to
> Docker/Mesos/Kubernetes etc. Almost without exception those systems work
> with configuration stored in text files (property, json, yaml etc), often
> stored in Git/SVN/Hg, edited by a human in a text editor. There are systems
> that only allow config via a GUI, but most of those only used, under
> sufferance, when there is no other option, (or competing product). The
> "customer" here is not a Sling/AEM expert, but a DevOps engineer who has
> never seen either before.
>
> If the only way to configure Sling/AEM is via special Sling/AEM tooling
> then we will have made the job of the TechOps/DevOps team even harder.
>
> I can think of lots of ways that merging at the property level could be
> made to work, but if we are not able to communicate clearly how it works,
> and all the potential side effects, to those whose job it is to make
> Sling/AEM work in production then we will have failed. If we have to resort
> to validation tools, compilers, special editors thats an indication the
> system is so complex we are not able to document it in plain english. For
> that reason, I beg, please keep whatever solution is chosen as simple as
> possible.
>
> I did try to document "object" level inheritance (ie aggregating
> properties), with all the side effects, but when I realised it was not
> going to be simple, I gave up and went back "object" level merging, which
> is brutal and simple by comparison.
>
> Best Regards
> Ian
>
>
>
> On 1 August 2016 at 08:29, Carsten Ziegeler <cz...@apache.org> wrote:
>
>>> Hi everyone,
>>>
>>> I personally see a tooling based approach complicated due to the
>> ownership
>>> scenarios.
>> Let's assume two scenarios, configuration A and configuration B inherits
>> from A.
>>
>> Now, if we support merging, someone is allowed to change A, adds a new
>> property and this automatically influences B - and in this case the
>> person who made changes to A does not even need to have access rights to
>> change anything for B. (Which I think is scary).
>>
>> If we don't support inheritance, there is still someone who is allowed
>> to make a change to A - at first this has no influence on B. If that
>> same person is allowed to also change B and wants to "inherit" the new
>> property to B, the tooling will allow so.
>>
>> With that respect, not having automatic inheritance sounds like the
>> better option when it comes to access rights.
>>
>> Carsten
>>
>>> We are talking about context aware configuration that typically
>>> would be applied by some runtime user that might have limited write
>> access
>>> (just for a specific context). The assumption that who writes to the
>>> inheriting configuration node also has write access to the subsequent
>>> levels is something we cannot and should not make here. I know multiple
>>> customer scenarios where a central site (including global configuration)
>>> was maintained by a central marketing organization while the markets
>> owned
>>> their own sites - so they declare inheritance (pull) instead of having
>> the
>>> global one pushed.
>>>
>>> The one thing in between that I could imagine might work sufficiently is
>>> referencing (by path to property) instead of inheritance. This still
>>> provides the lookup of "inheritance" at runtime where changes are
>> reflected
>>> in the lookup immediately but at the same time gets rid of magic
>>> inheritance rules.
>>>
>>> Just a thought about an alternative in between.
>>>
>>> Cheers
>>> Dominik
>>>
>>> On Mon, Aug 1, 2016 at 8:36 AM, Carsten Ziegeler <cz...@apache.org>
>>> wrote:
>>>
>>>>>> ... if you have merging (or inheritance) then you change the
>>>>>> attribute of a parent and this influences all childs and you
>>>>>> have no idea what happens there...
>>>>>
>>>>> We can create a felix console plugin that shows for a given
>>>>> context path and config name the value of the property and the
>>>>> path from where it was taken - that way both developers and ops
>>>>> can quickly check what's going wrong if results are unexpected.
>>>>>
>>>>>> I personally think this comes all down to tooling ...
>>>>>
>>>>> So this could mean e.g. an additional maven plugin, which handles
>>>>> merging of configurations and then the runtime can work without
>>>>> merging. The problem I see with that is that
>>>>> * you end up having two similar configuration model types (one in
>>>>> the source code that supports merging and one effective one for
>>>>> the runtime) - this makes the mechanism harder to understand
>>>>> for everyone
>>>>> * the tooling has to be created and IMHO it'll not be easier/less
>>>>> code than creating a smarter runtime (even if we take into
>>>>> account that we have to create the felix console plugin to make
>>>>> it traceable)
>>>>>
>>>> First of all, you need tooling anyway. It would be foolish to make
>>>> these configurations without any tooling by hand. Tooling can easily
>>>> be customized to anyones needs without tampering the implementation.
>>>> The implementation at runtime can be simply and effective.
>>>>
>>>> We had similar discussions while starting the work for the OSGi
>>>> Configurator which allows you to put OSGi configurations into a bundle
>>>> which then get applied at runtime. My initial proposal had all the
>>>> things requested here as well (merging, inheritance etc.). It turned
>>>> out that this messed up the specification and made the implementation
>>>> extraordinary complicated. While moving this into the tooling which
>>>> creates the bundles in the first place solves all the problems.
>>>>
>>>> Just think of all the different cases you have to solve once you
>>>> start support merging. And not to mention the runtime implications as
>>>> you have to do the merging dynamically at runtime as well, or you
>>>> start introducing all kinds of caching again. All of this is not
>>>> required if you move this to the tooling level.
>>>>
>>>> Regards
>>>>
>>>> Carsten
>>>>
>>>> --
>>>> Carsten Ziegeler
>>>> Adobe Research Switzerland
>>>> cziegeler@apache.org
>>>>
>>>>
>>>
>>
>>
>>
>>
>> --
>> Carsten Ziegeler
>> Adobe Research Switzerland
>> cziegeler@apache.org
>>
>>
>
--
Carsten Ziegeler
Adobe Research Switzerland
cziegeler@apache.org
Re: [Context Aware Configs] Merging?
Posted by Ian Boston <ie...@tfd.co.uk>.
Hi,
I think it must be possible to configure a Sling/AEM instance without using
specific Sling/AEM tooling to create and manage the source configuration.
The reason I say this is almost all serious deployments use some form of
DevOps automation ranging from Puppet/Chef/AWS etc through to
Docker/Mesos/Kubernetes etc. Almost without exception those systems work
with configuration stored in text files (property, json, yaml etc), often
stored in Git/SVN/Hg, edited by a human in a text editor. There are systems
that only allow config via a GUI, but most of those only used, under
sufferance, when there is no other option, (or competing product). The
"customer" here is not a Sling/AEM expert, but a DevOps engineer who has
never seen either before.
If the only way to configure Sling/AEM is via special Sling/AEM tooling
then we will have made the job of the TechOps/DevOps team even harder.
I can think of lots of ways that merging at the property level could be
made to work, but if we are not able to communicate clearly how it works,
and all the potential side effects, to those whose job it is to make
Sling/AEM work in production then we will have failed. If we have to resort
to validation tools, compilers, special editors thats an indication the
system is so complex we are not able to document it in plain english. For
that reason, I beg, please keep whatever solution is chosen as simple as
possible.
I did try to document "object" level inheritance (ie aggregating
properties), with all the side effects, but when I realised it was not
going to be simple, I gave up and went back "object" level merging, which
is brutal and simple by comparison.
Best Regards
Ian
On 1 August 2016 at 08:29, Carsten Ziegeler <cz...@apache.org> wrote:
> > Hi everyone,
> >
> > I personally see a tooling based approach complicated due to the
> ownership
> > scenarios.
> Let's assume two scenarios, configuration A and configuration B inherits
> from A.
>
> Now, if we support merging, someone is allowed to change A, adds a new
> property and this automatically influences B - and in this case the
> person who made changes to A does not even need to have access rights to
> change anything for B. (Which I think is scary).
>
> If we don't support inheritance, there is still someone who is allowed
> to make a change to A - at first this has no influence on B. If that
> same person is allowed to also change B and wants to "inherit" the new
> property to B, the tooling will allow so.
>
> With that respect, not having automatic inheritance sounds like the
> better option when it comes to access rights.
>
> Carsten
>
> > We are talking about context aware configuration that typically
> > would be applied by some runtime user that might have limited write
> access
> > (just for a specific context). The assumption that who writes to the
> > inheriting configuration node also has write access to the subsequent
> > levels is something we cannot and should not make here. I know multiple
> > customer scenarios where a central site (including global configuration)
> > was maintained by a central marketing organization while the markets
> owned
> > their own sites - so they declare inheritance (pull) instead of having
> the
> > global one pushed.
> >
> > The one thing in between that I could imagine might work sufficiently is
> > referencing (by path to property) instead of inheritance. This still
> > provides the lookup of "inheritance" at runtime where changes are
> reflected
> > in the lookup immediately but at the same time gets rid of magic
> > inheritance rules.
> >
> > Just a thought about an alternative in between.
> >
> > Cheers
> > Dominik
> >
> > On Mon, Aug 1, 2016 at 8:36 AM, Carsten Ziegeler <cz...@apache.org>
> > wrote:
> >
> >>>> ... if you have merging (or inheritance) then you change the
> >>>> attribute of a parent and this influences all childs and you
> >>>> have no idea what happens there...
> >>>
> >>> We can create a felix console plugin that shows for a given
> >>> context path and config name the value of the property and the
> >>> path from where it was taken - that way both developers and ops
> >>> can quickly check what's going wrong if results are unexpected.
> >>>
> >>>> I personally think this comes all down to tooling ...
> >>>
> >>> So this could mean e.g. an additional maven plugin, which handles
> >>> merging of configurations and then the runtime can work without
> >>> merging. The problem I see with that is that
> >>> * you end up having two similar configuration model types (one in
> >>> the source code that supports merging and one effective one for
> >>> the runtime) - this makes the mechanism harder to understand
> >>> for everyone
> >>> * the tooling has to be created and IMHO it'll not be easier/less
> >>> code than creating a smarter runtime (even if we take into
> >>> account that we have to create the felix console plugin to make
> >>> it traceable)
> >>>
> >> First of all, you need tooling anyway. It would be foolish to make
> >> these configurations without any tooling by hand. Tooling can easily
> >> be customized to anyones needs without tampering the implementation.
> >> The implementation at runtime can be simply and effective.
> >>
> >> We had similar discussions while starting the work for the OSGi
> >> Configurator which allows you to put OSGi configurations into a bundle
> >> which then get applied at runtime. My initial proposal had all the
> >> things requested here as well (merging, inheritance etc.). It turned
> >> out that this messed up the specification and made the implementation
> >> extraordinary complicated. While moving this into the tooling which
> >> creates the bundles in the first place solves all the problems.
> >>
> >> Just think of all the different cases you have to solve once you
> >> start support merging. And not to mention the runtime implications as
> >> you have to do the merging dynamically at runtime as well, or you
> >> start introducing all kinds of caching again. All of this is not
> >> required if you move this to the tooling level.
> >>
> >> Regards
> >>
> >> Carsten
> >>
> >> --
> >> Carsten Ziegeler
> >> Adobe Research Switzerland
> >> cziegeler@apache.org
> >>
> >>
> >
>
>
>
>
> --
> Carsten Ziegeler
> Adobe Research Switzerland
> cziegeler@apache.org
>
>
Re: [Context Aware Configs] Merging?
Posted by Carsten Ziegeler <cz...@apache.org>.
> Hi everyone,
>
> I personally see a tooling based approach complicated due to the ownership
> scenarios.
Let's assume two scenarios, configuration A and configuration B inherits
from A.
Now, if we support merging, someone is allowed to change A, adds a new
property and this automatically influences B - and in this case the
person who made changes to A does not even need to have access rights to
change anything for B. (Which I think is scary).
If we don't support inheritance, there is still someone who is allowed
to make a change to A - at first this has no influence on B. If that
same person is allowed to also change B and wants to "inherit" the new
property to B, the tooling will allow so.
With that respect, not having automatic inheritance sounds like the
better option when it comes to access rights.
Carsten
> We are talking about context aware configuration that typically
> would be applied by some runtime user that might have limited write access
> (just for a specific context). The assumption that who writes to the
> inheriting configuration node also has write access to the subsequent
> levels is something we cannot and should not make here. I know multiple
> customer scenarios where a central site (including global configuration)
> was maintained by a central marketing organization while the markets owned
> their own sites - so they declare inheritance (pull) instead of having the
> global one pushed.
>
> The one thing in between that I could imagine might work sufficiently is
> referencing (by path to property) instead of inheritance. This still
> provides the lookup of "inheritance" at runtime where changes are reflected
> in the lookup immediately but at the same time gets rid of magic
> inheritance rules.
>
> Just a thought about an alternative in between.
>
> Cheers
> Dominik
>
> On Mon, Aug 1, 2016 at 8:36 AM, Carsten Ziegeler <cz...@apache.org>
> wrote:
>
>>>> ... if you have merging (or inheritance) then you change the
>>>> attribute of a parent and this influences all childs and you
>>>> have no idea what happens there...
>>>
>>> We can create a felix console plugin that shows for a given
>>> context path and config name the value of the property and the
>>> path from where it was taken - that way both developers and ops
>>> can quickly check what's going wrong if results are unexpected.
>>>
>>>> I personally think this comes all down to tooling ...
>>>
>>> So this could mean e.g. an additional maven plugin, which handles
>>> merging of configurations and then the runtime can work without
>>> merging. The problem I see with that is that
>>> * you end up having two similar configuration model types (one in
>>> the source code that supports merging and one effective one for
>>> the runtime) - this makes the mechanism harder to understand
>>> for everyone
>>> * the tooling has to be created and IMHO it'll not be easier/less
>>> code than creating a smarter runtime (even if we take into
>>> account that we have to create the felix console plugin to make
>>> it traceable)
>>>
>> First of all, you need tooling anyway. It would be foolish to make
>> these configurations without any tooling by hand. Tooling can easily
>> be customized to anyones needs without tampering the implementation.
>> The implementation at runtime can be simply and effective.
>>
>> We had similar discussions while starting the work for the OSGi
>> Configurator which allows you to put OSGi configurations into a bundle
>> which then get applied at runtime. My initial proposal had all the
>> things requested here as well (merging, inheritance etc.). It turned
>> out that this messed up the specification and made the implementation
>> extraordinary complicated. While moving this into the tooling which
>> creates the bundles in the first place solves all the problems.
>>
>> Just think of all the different cases you have to solve once you
>> start support merging. And not to mention the runtime implications as
>> you have to do the merging dynamically at runtime as well, or you
>> start introducing all kinds of caching again. All of this is not
>> required if you move this to the tooling level.
>>
>> Regards
>>
>> Carsten
>>
>> --
>> Carsten Ziegeler
>> Adobe Research Switzerland
>> cziegeler@apache.org
>>
>>
>
--
Carsten Ziegeler
Adobe Research Switzerland
cziegeler@apache.org
Re: [Context Aware Configs] Merging?
Posted by Dominik Süß <do...@gmail.com>.
Hi everyone,
I personally see a tooling based approach complicated due to the ownership
scenarios. We are talking about context aware configuration that typically
would be applied by some runtime user that might have limited write access
(just for a specific context). The assumption that who writes to the
inheriting configuration node also has write access to the subsequent
levels is something we cannot and should not make here. I know multiple
customer scenarios where a central site (including global configuration)
was maintained by a central marketing organization while the markets owned
their own sites - so they declare inheritance (pull) instead of having the
global one pushed.
The one thing in between that I could imagine might work sufficiently is
referencing (by path to property) instead of inheritance. This still
provides the lookup of "inheritance" at runtime where changes are reflected
in the lookup immediately but at the same time gets rid of magic
inheritance rules.
Just a thought about an alternative in between.
Cheers
Dominik
On Mon, Aug 1, 2016 at 8:36 AM, Carsten Ziegeler <cz...@apache.org>
wrote:
> >> ... if you have merging (or inheritance) then you change the
> >> attribute of a parent and this influences all childs and you
> >> have no idea what happens there...
> >
> > We can create a felix console plugin that shows for a given
> > context path and config name the value of the property and the
> > path from where it was taken - that way both developers and ops
> > can quickly check what's going wrong if results are unexpected.
> >
> >> I personally think this comes all down to tooling ...
> >
> > So this could mean e.g. an additional maven plugin, which handles
> > merging of configurations and then the runtime can work without
> > merging. The problem I see with that is that
> > * you end up having two similar configuration model types (one in
> > the source code that supports merging and one effective one for
> > the runtime) - this makes the mechanism harder to understand
> > for everyone
> > * the tooling has to be created and IMHO it'll not be easier/less
> > code than creating a smarter runtime (even if we take into
> > account that we have to create the felix console plugin to make
> > it traceable)
> >
> First of all, you need tooling anyway. It would be foolish to make
> these configurations without any tooling by hand. Tooling can easily
> be customized to anyones needs without tampering the implementation.
> The implementation at runtime can be simply and effective.
>
> We had similar discussions while starting the work for the OSGi
> Configurator which allows you to put OSGi configurations into a bundle
> which then get applied at runtime. My initial proposal had all the
> things requested here as well (merging, inheritance etc.). It turned
> out that this messed up the specification and made the implementation
> extraordinary complicated. While moving this into the tooling which
> creates the bundles in the first place solves all the problems.
>
> Just think of all the different cases you have to solve once you
> start support merging. And not to mention the runtime implications as
> you have to do the merging dynamically at runtime as well, or you
> start introducing all kinds of caching again. All of this is not
> required if you move this to the tooling level.
>
> Regards
>
> Carsten
>
> --
> Carsten Ziegeler
> Adobe Research Switzerland
> cziegeler@apache.org
>
>
Re: [Context Aware Configs] Merging?
Posted by Carsten Ziegeler <cz...@apache.org>.
>> ... if you have merging (or inheritance) then you change the
>> attribute of a parent and this influences all childs and you
>> have no idea what happens there...
>
> We can create a felix console plugin that shows for a given
> context path and config name the value of the property and the
> path from where it was taken - that way both developers and ops
> can quickly check what's going wrong if results are unexpected.
>
>> I personally think this comes all down to tooling ...
>
> So this could mean e.g. an additional maven plugin, which handles
> merging of configurations and then the runtime can work without
> merging. The problem I see with that is that
> * you end up having two similar configuration model types (one in
> the source code that supports merging and one effective one for
> the runtime) - this makes the mechanism harder to understand
> for everyone
> * the tooling has to be created and IMHO it'll not be easier/less
> code than creating a smarter runtime (even if we take into
> account that we have to create the felix console plugin to make
> it traceable)
>
First of all, you need tooling anyway. It would be foolish to make
these configurations without any tooling by hand. Tooling can easily
be customized to anyones needs without tampering the implementation.
The implementation at runtime can be simply and effective.
We had similar discussions while starting the work for the OSGi
Configurator which allows you to put OSGi configurations into a bundle
which then get applied at runtime. My initial proposal had all the
things requested here as well (merging, inheritance etc.). It turned
out that this messed up the specification and made the implementation
extraordinary complicated. While moving this into the tooling which
creates the bundles in the first place solves all the problems.
Just think of all the different cases you have to solve once you
start support merging. And not to mention the runtime implications as
you have to do the merging dynamically at runtime as well, or you
start introducing all kinds of caching again. All of this is not
required if you move this to the tooling level.
Regards
Carsten
--
Carsten Ziegeler
Adobe Research Switzerland
cziegeler@apache.org