UC: Combining PropertyProvider Instances

[Anatole Tresch <an...@apache.org>]

   Hi all
find following an additional part of use cases for comment. Note that this
area IMO definitively may be improved in several ways. Always keep in mind
that I try to setup the design guide similarly in parallel. It can be
easily accessed from the GH mirror:

https://github.com/apache/incubator-tamaya/blob/master/docs/design/2_CoreConcepts.adoc

Cheers,
Anatole

Combining Property Providers

Looking at the structures of configuration system used by large companies
we typically encounter some kind of configuration hierarchies that are
combined in arbitrary ways. Users of the systems are typically not aware of
the complexities in this area, since they simply know the possible
locations, formats and the overriding policies. Framework providers on the
other side must face the complexities and it would be very useful if Tamaya
can support here by providing prebuilt functionality that helps
implementing these aspects. All this leads to the feature set of combining
property providers. Hereby the following strategies are useful:

   -

   aggregating providers, hereby later providers added
    -

      override any existing entries from earlier providers
       -

      combine conflicting entries from earlier providers, e.g. into a
      comma-separated structure.
       -

      may throw a ConfigExcepotion ig entries are conflicting
       -

      may only add entries not yet defined by former providers, preventing
      entries that are already present to be overwritte
       -

      any custom aggregation strategy, which may be a mix of above
        -

   intersecting providers
    -

   subtracting providers
    -

   filtering providers

  These common functionality is provided by the PropertyProviders
singleton. Additionally to the base strategies above a MetaInfo instance
can be passed optionally as well to define the meta information for the
newly created provider instances. Let’s assume we have two property
providers with the following data:
  Provider 1

a=a
b=b
c=c
g=g
h=h
i=i

  Provider 2

a=A
b=B
c=C
d=D
e=E
f=F

  Looking in detail you see that the entries a,b,c are present in both
providers, whereas d,e,f are only present in provider 1, and g,h,i only in
provider 2.
  Example Combining PropertyProviders

PropertyProvider provider1 = ...
PropertyProvider provider2 = ...
// aggregate, hereby values from provider 2 override values from provider 1
PropertyProvider unionOverriding =
PropertyProviders.aggregate(AggregationPolicy.OVERRIDE(), provider1,
provider2);*System*.out.println("unionOverriding: " +
unionOverriding);
// ignore duplicates, values present in provider 1 are not overriden
by provider 2
PropertyProvider unionIgnoringDuplicates =
PropertyProviders.aggregate(AggregationPolicy.IGNORE_DUPLICATES(),
provider1, provider2);*System*.out.println("unionIgnoringDuplicates: "
+ unionIgnoringDuplicates);
// this variant combines/maps duplicate values into a new value
PropertyProvider unionCombined =
PropertyProviders.aggregate(AggregationPolicy.COMBINE(), provider1,
provider2);*System*.out.println("unionCombined: " + unionCombined);
// This variant throws an exception since there are key/value paris in
both providers, but with different values*try*{
    PropertyProviders.aggregate(AggregationPolicy.EXCEPTION(),
provider1, provider2);
}*catch*(ConfigException e){
    // expected!
}

  The example above produces the following outpout:
  Example Combining PropertyProviders

AggregatedPropertyProvider{
  (name = dynamicAggregationTests)
  a = "[a][A]"
  b = "[b][B]"
  c = "[c][C]"
  d = "[D]"
  e = "[E]"
  f = "[F]"
  g = "[g]"
  h = "[h]"
  i = "[i]"
}
unionOverriding: AggregatedPropertyProvider{
  (name = <noname>)
  a = "A"
  b = "B"
  c = "C"
  d = "D"
  e = "E"
  f = "F"
  g = "g"
  h = "h"
  i = "i"
}
unionIgnoringDuplicates: AggregatedPropertyProvider{
  (name = <noname>)
  a = "a"
  b = "b"
  c = "c"
  d = "D"
  e = "E"
  f = "F"
  g = "g"
  h = "h"
  i = "i"
}
unionCombined: AggregatedPropertyProvider{
  (name = <noname>)
  a = "a,A"
  b = "b,B"
  c = "c,C"
  d = "D"
  e = "E"
  f = "F"
  g = "g"
  h = "h"
  i = "i"
}

  No AggregationPolicy is also an interface that can be implemented:
  AggregationPolicy Interface

@FunctionalInterface*public* *interface* *AggregationPolicy* {
    *String* aggregate(*String* key, *String* value1, *String* value2);
}

  So we can also define our own aggregation strategy using a Lambda
expression:
  Use a Custom AggregationPolicy

PropertyProvider provider1 = ...;
PropertyProvider provider2 = ...;
PropertyProvider props = PropertyProviders.aggregate(
      (k, v1, v2) -> (v1 != null ? v1 : "") + '[' + v2 + "]",
      MetaInfo.of("dynamicAggregationTests"),
      props1, props2);*System*.out.println(props);

  Additionally we also pass here an instance of MetaInfo. The output of
this code snippet is as follows:
  Listing of dynamic aggregation policy

AggregatedPropertyProvider{
  (name = dynamicAggregationTests)
  a = "[a][A]"
  b = "[b][B]"
  c = "[c][C]"
  d = "[D]"
  e = "[E]"
  f = "[F]"
  g = "[g]"
  h = "[h]"
  i = "[i]"
}

  Summarizing the PropertyProviders singleton allows to combine providers
in various forms:
  Methods provided on PropertyProviders

public final class PropertyProviders {

    private PropertyProviders() {}

    public static PropertyProvider fromArgs(String... args) {
    public static PropertyProvider fromArgs(MetaInfo metaInfo, String... args) {
    public static PropertyProvider fromPaths(AggregationPolicy
aggregationPolicy, String... paths) {
    public static PropertyProvider fromPaths(String... paths) {
    public static PropertyProvider fromPaths(List<String> paths) {
    public static PropertyProvider fromPaths(AggregationPolicy
aggregationPolicy, List<String> paths) {
    public static PropertyProvider fromPaths(MetaInfo metaInfo,
List<String> paths) {
    public static PropertyProvider fromPaths(AggregationPolicy
aggregationPolicy, MetaInfo metaInfo, List<String> paths) {
    public static PropertyProvider fromUris(URI... uris) {
    public static PropertyProvider fromUris(AggregationPolicy
aggregationPolicy, URI... uris) {
    public static PropertyProvider fromUris(List<URI> uris) {
    public static PropertyProvider fromUris(AggregationPolicy
aggregationPolicy, List<URI> uris) {
    public static PropertyProvider fromUris(MetaInfo metaInfo, URI... uris) {
    public static PropertyProvider fromUris(AggregationPolicy
aggregationPolicy, MetaInfo metaInfo, URI... uris) {
    public static PropertyProvider fromUris(MetaInfo metaInfo, List<URI> uris) {
    public static PropertyProvider fromUris(AggregationPolicy
aggregationPolicy, MetaInfo metaInfo, List<URI> uris) {
    public static PropertyProvider fromMap(Map<String, String> map) {
    public static PropertyProvider fromMap(MetaInfo metaInfo,
Map<String, String> map) {
    public static PropertyProvider empty() {
    public static PropertyProvider emptyMutable() {
    public static PropertyProvider empty(MetaInfo metaInfo) {
    public static PropertyProvider emptyMutable(MetaInfo metaInfo) {
    public static PropertyProvider fromEnvironmentProperties() {
    public static PropertyProvider fromSystemProperties() {
    public static PropertyProvider freezed(PropertyProvider provider) {
    public static PropertyProvider aggregate(AggregationPolicy
mapping, MetaInfo metaInfo, PropertyProvider... providers){
    public static PropertyProvider aggregate(PropertyProvider... providers) {
    public static PropertyProvider aggregate(List<PropertyProvider> providers) {
    public static PropertyProvider aggregate(AggregationPolicy
mapping, PropertyProvider... propertyMaps) {
    public static PropertyProvider aggregate(AggregationPolicy
mapping, List<PropertyProvider> providers) {
    public static PropertyProvider mutable(PropertyProvider provider) {
    public static PropertyProvider intersected(AggregationPolicy
aggregationPolicy, PropertyProvider... providers) {
    public static PropertyProvider intersected(PropertyProvider... providers) {
    public static PropertyProvider subtracted(PropertyProvider target,
PropertyProvider... providers) {
    public static PropertyProvider filtered(Predicate<String> filter,
PropertyProvider provider) {
    public static PropertyProvider
contextual(Supplier<PropertyProvider> mapSupplier,
                                              Supplier<String>
isolationKeySupplier) {
    public static PropertyProvider delegating(PropertyProvider
mainMap, Map<String, String> parentMap) {
    public static PropertyProvider replacing(PropertyProvider mainMap,
Map<String, String> replacementMap) {
}

Re: UC: Combining PropertyProvider Instances

[Anatole Tresch <at...@gmail.com>]

Tbd. As of now my idea was to use Java to define the meta level. Or
whatever language is running on the jvm with tamaya...
But the corresponding concepts/ideas are not yet/only partially there as of
now...
Werner Keil <we...@gmail.com> schrieb am Do., 4. Dez. 2014 um 15:20:

> While using XML a lot of it can be found that Multiconf did in Python.
>
> (I am pretty sure most of Spring Config still uses plenty of XML, too, in
> theory similar to Multiconf it could use a Groovy DSL, maybe if Pivotal
> used Tamaya some day and the Groovy experts here considered such DSL a
> worthy effort?;-D)
>
>
>
>
> On Thu, Dec 4, 2014 at 2:53 PM, Tresch, Anatole <
> anatole.tresch@credit-suisse.com> wrote:
>
> > Hi Mark
> >
> > let me do a last try to clarifiy my ideas, I am open to discuss things
> > (and do add some ideas at the end, so we can start the discussions right
> > away ;) ):
> >
> > 1) my idea was to define a minimalistic interface for properties first.
> It
> > should as simple as possible to implement. Just using Map<String,String>
> I
> > found not optimal, since a Map requires several methods that are not easy
> > to implement in all cases, such as size(). Also Spring provides such a
> > similarl abstraction(PropertySource). For this I created
> PropertyProvider.
> > 2) Looking at Configuration there is much more functionality needed to
> > have a powerful config API, such as type support (getInt, getBoolean,
> > getAdapted), registering of listeners, extension points (with, query).
> From
> > our bank
> >    additional methods were added for evaluation of the key ranges
> present,
> > assuming hereby a.b.d: a is an area, containing an area b containing a
> key
> > d. So I defined the Configuration interface.
> > 3) Now there are very common types of properties sources you want to read
> > from. I personally identified: classpath resources, files and URLs/URIs.
> So
> > I added some kind of factory class (PropertyProviders), which allows to
> > easily
> >    construct them descriptively: PropertyProvider proc =
> > PropertyProviders.fromPath("classpath:META-INF/cfg/config.ini");
> > 4) In many companies you do not have a fixed configuration file name, you
> > will have locations, where you consume all files (even recursively). So
> > adding resource loader support similar to Spring was compelling:
> >     PropertyProvider proc =
> > PropertyProviders.fromPath("classpath:META-INF/cfg/**/*.ini");
> > 5) Looking at what we have in Credit Suisse and what other prople told
> > they have in their companies, simple priority based adding of maps is not
> > enough. There are cases, where I want to filter out values, prevent
> > overriding of certain values etc. Also different locations (classpath,
> > files, remote) are mixed and depending on the source different priorities
> > may be assigned. Finally configuration pairs themselves can be attributed
> > (default entries, global default entries, explicit entries, ..), which
> also
> > lead to different configuration priorities within the same source: or
> even
> > config file, e.g:
> >
> > <config>
> >   <defaults>
> >    <a>aValue</a>
> >   <b>bValue</b>
> > </deault>
> >
> > <pluginConfig plugin="abc">
> >   <a>aValue2</a>
> > </pluginConfig>
> >
> > <server ip="1.2.4.5">
> >      <b>bValue2</b>
> > </server>
> > </config>
> >
> > Since I saw definitively the most complex scenarios I wanted to support
> > power users doing that kind of modelling more easily. One mechanism to
> > achive this is allowing to aggregate and combine maps (=providers), so
> > I added the combination methods on PropertyProviders like aggregate,
> > intersect, filter, ...
> > Now this seems more complicated for you guys than it seems for me ;) I
> can
> > imagine different things that may be simplified:
> >
> > - Remove PropertyProvider/ PropertyProviders from the API part and just
> > add corresponding functionality using a ConfigurationBuilder:
> >
> >   Configuration config =
> > ConfigurationBuilder.create("myConfig").addPaths("ab/B/B",
> > "../fhgg.xml").with(AggregationPolicy.OVERRIDE).
> addPaths("jhbjh/kjhkjh/").build();
> >   Config aggregatedConfig = config.with(Aggregators.from(config);
> >
> > - We still could put the PropertyProvider to the core implementation
> > module, or even add it as a separate module, for the ones that like a
> more
> > leightweight implementation SPI ;)
> >
> > Configuration conf = Configuration.current(); will still be the accessor
> > method of choice. Though I am well aware of the fact that the
> abstractions
> > (SPIs) required to evaluate the right configuration during runtime  is
> not
> > yet discussed. Also the current state in the repo in that area is far
> away
> > from finished (I currently try to cleanup a few things, especially
> > artifacts that are duplicates...).
> >
> > Best,
> > Anatole
> >
> >
> >
> > -----Original Message-----
> > From: Mark Struberg [mailto:struberg@yahoo.de]
> > Sent: Donnerstag, 4. Dezember 2014 11:08
> > To: dev@tamaya.incubator.apache.org
> > Subject: Re: UC: Combining PropertyProvider Instances
> >
> > I think I need a bit time to understand this. I've tried to grasp it for
> 2
> > days now and this might mean that it is too complicated if not even I get
> > it. Why are we doing this so complicated? Pulling in all the database
> logic
> > into a tool which main target is that it should be really easy to use?
> > What benefit does this add over hiding those details? e.g. in DeltaSpike
> I
> > would just add a DatabaseConfigSource and be done. The user would not
> even
> > see that it magically picks up the values from the db then.
> >
> >
> > LieGrue,
> > strub
> >
> >
> >
> >
> >
> > > On Tuesday, 2 December 2014, 14:02, Anatole Tresch <anatole@apache.org
> >
> > wrote:
> > > >    Hi all
> > > find following an additional part of use cases for comment. Note that
> > this
> > > area IMO definitively may be improved in several ways. Always keep in
> > mind
> > > that I try to setup the design guide similarly in parallel. It can be
> > > easily accessed from the GH mirror:
> > >
> > >
> > https://github.com/apache/incubator-tamaya/blob/master/
> docs/design/2_CoreConcepts.adoc
> > >
> > > Cheers,
> > > Anatole
> > >
> > > Combining Property Providers
> > >
> > > Looking at the structures of configuration system used by large
> companies
> > > we typically encounter some kind of configuration hierarchies that are
> > > combined in arbitrary ways. Users of the systems are typically not
> aware
> > of
> > > the complexities in this area, since they simply know the possible
> > > locations, formats and the overriding policies. Framework providers on
> > the
> > > other side must face the complexities and it would be very useful if
> > Tamaya
> > > can support here by providing prebuilt functionality that helps
> > > implementing these aspects. All this leads to the feature set of
> > combining
> > > property providers. Hereby the following strategies are useful:
> > >
> > >    -
> > >
> > >    aggregating providers, hereby later providers added
> > >     -
> > >
> > >       override any existing entries from earlier providers
> > >        -
> > >
> > >       combine conflicting entries from earlier providers, e.g. into a
> > >       comma-separated structure.
> > >        -
> > >
> > >       may throw a ConfigExcepotion ig entries are conflicting
> > >        -
> > >
> > >       may only add entries not yet defined by former providers,
> > preventing
> > >       entries that are already present to be overwritte
> > >        -
> > >
> > >       any custom aggregation strategy, which may be a mix of above
> > >         -
> > >
> > >    intersecting providers
> > >     -
> > >
> > >    subtracting providers
> > >     -
> > >
> > >    filtering providers
> > >
> > >   These common functionality is provided by the PropertyProviders
> > > singleton. Additionally to the base strategies above a MetaInfo
> instance
> > > can be passed optionally as well to define the meta information for the
> > > newly created provider instances. Let’s assume we have two property
> > > providers with the following data:
> > >   Provider 1
> > >
> > > a=a
> > > b=b
> > > c=c
> > > g=g
> > > h=h
> > > i=i
> > >
> > >   Provider 2
> > >
> > > a=A
> > > b=B
> > > c=C
> > > d=D
> > > e=E
> > > f=F
> > >
> > >   Looking in detail you see that the entries a,b,c are present in both
> > > providers, whereas d,e,f are only present in provider 1, and g,h,i only
> > in
> > > provider 2.
> > >   Example Combining PropertyProviders
> > >
> > > PropertyProvider provider1 = ...
> > > PropertyProvider provider2 = ...
> > > // aggregate, hereby values from provider 2 override values from
> > provider 1
> > > PropertyProvider unionOverriding =
> > > PropertyProviders.aggregate(AggregationPolicy.OVERRIDE(), provider1,
> > > provider2);*System*.out.println("unionOverriding: " +
> > > unionOverriding);
> > > // ignore duplicates, values present in provider 1 are not overriden
> > > by provider 2
> > > PropertyProvider unionIgnoringDuplicates =
> > > PropertyProviders.aggregate(AggregationPolicy.IGNORE_DUPLICATES(),
> > > provider1, provider2);*System*.out.println("unionIgnoringDuplicates: "
> > > + unionIgnoringDuplicates);
> > > // this variant combines/maps duplicate values into a new value
> > > PropertyProvider unionCombined =
> > > PropertyProviders.aggregate(AggregationPolicy.COMBINE(), provider1,
> > > provider2);*System*.out.println("unionCombined: " + unionCombined);
> > > // This variant throws an exception since there are key/value paris in
> > > both providers, but with different values*try*{
> > >     PropertyProviders.aggregate(AggregationPolicy.EXCEPTION(),
> > > provider1, provider2);
> > > }*catch*(ConfigException e){
> > >     // expected!
> > > }
> > >
> > >   The example above produces the following outpout:
> > >   Example Combining PropertyProviders
> > >
> > > AggregatedPropertyProvider{
> > >   (name = dynamicAggregationTests)
> > >   a = "[a][A]"
> > >   b = "[b][B]"
> > >   c = "[c][C]"
> > >   d = "[D]"
> > >   e = "[E]"
> > >   f = "[F]"
> > >   g = "[g]"
> > >   h = "[h]"
> > >   i = "[i]"
> > > }
> > > unionOverriding: AggregatedPropertyProvider{
> > >   (name = <noname>)
> > >   a = "A"
> > >   b = "B"
> > >   c = "C"
> > >   d = "D"
> > >   e = "E"
> > >   f = "F"
> > >   g = "g"
> > >   h = "h"
> > >   i = "i"
> > > }
> > > unionIgnoringDuplicates: AggregatedPropertyProvider{
> > >   (name = <noname>)
> > >   a = "a"
> > >   b = "b"
> > >   c = "c"
> > >   d = "D"
> > >   e = "E"
> > >   f = "F"
> > >   g = "g"
> > >   h = "h"
> > >   i = "i"
> > > }
> > > unionCombined: AggregatedPropertyProvider{
> > >   (name = <noname>)
> > >   a = "a,A"
> > >   b = "b,B"
> > >   c = "c,C"
> > >   d = "D"
> > >   e = "E"
> > >   f = "F"
> > >   g = "g"
> > >   h = "h"
> > >   i = "i"
> > > }
> > >
> > >   No AggregationPolicy is also an interface that can be implemented:
> > >   AggregationPolicy Interface
> > >
> > > @FunctionalInterface*public* *interface* *AggregationPolicy* {
> > >     *String* aggregate(*String* key, *String* value1, *String* value2);
> > > }
> > >
> > >   So we can also define our own aggregation strategy using a Lambda
> > > expression:
> > >   Use a Custom AggregationPolicy
> > >
> > > PropertyProvider provider1 = ...;
> > > PropertyProvider provider2 = ...;
> > > PropertyProvider props = PropertyProviders.aggregate(
> > >       (k, v1, v2) -> (v1 != null ? v1 : "") + '[' + v2 +
> > > "]",
> > >       MetaInfo.of("dynamicAggregationTests"),
> > >       props1, props2);*System*.out.println(props);
> > >
> > >   Additionally we also pass here an instance of MetaInfo. The output of
> > > this code snippet is as follows:
> > >   Listing of dynamic aggregation policy
> > >
> > > AggregatedPropertyProvider{
> > >   (name = dynamicAggregationTests)
> > >   a = "[a][A]"
> > >   b = "[b][B]"
> > >   c = "[c][C]"
> > >   d = "[D]"
> > >   e = "[E]"
> > >   f = "[F]"
> > >   g = "[g]"
> > >   h = "[h]"
> > >   i = "[i]"
> > > }
> > >
> > >   Summarizing the PropertyProviders singleton allows to combine
> providers
> > > in various forms:
> > >   Methods provided on PropertyProviders
> > >
> > > public final class PropertyProviders {
> > >
> > >     private PropertyProviders() {}
> > >
> > >     public static PropertyProvider fromArgs(String... args) {
> > >     public static PropertyProvider fromArgs(MetaInfo metaInfo,
> String...
> > args)
> > > {
> > >     public static PropertyProvider fromPaths(AggregationPolicy
> > > aggregationPolicy, String... paths) {
> > >     public static PropertyProvider fromPaths(String... paths) {
> > >     public static PropertyProvider fromPaths(List<String> paths) {
> > >     public static PropertyProvider fromPaths(AggregationPolicy
> > > aggregationPolicy, List<String> paths) {
> > >     public static PropertyProvider fromPaths(MetaInfo metaInfo,
> > > List<String> paths) {
> > >     public static PropertyProvider fromPaths(AggregationPolicy
> > > aggregationPolicy, MetaInfo metaInfo, List<String> paths) {
> > >     public static PropertyProvider fromUris(URI... uris) {
> > >     public static PropertyProvider fromUris(AggregationPolicy
> > > aggregationPolicy, URI... uris) {
> > >     public static PropertyProvider fromUris(List<URI> uris) {
> > >     public static PropertyProvider fromUris(AggregationPolicy
> > > aggregationPolicy, List<URI> uris) {
> > >     public static PropertyProvider fromUris(MetaInfo metaInfo, URI...
> > uris)
> > > {
> > >     public static PropertyProvider fromUris(AggregationPolicy
> > > aggregationPolicy, MetaInfo metaInfo, URI... uris) {
> > >     public static PropertyProvider fromUris(MetaInfo metaInfo,
> List<URI>
> > > uris) {
> > >     public static PropertyProvider fromUris(AggregationPolicy
> > > aggregationPolicy, MetaInfo metaInfo, List<URI> uris) {
> > >     public static PropertyProvider fromMap(Map<String, String> map) {
> > >     public static PropertyProvider fromMap(MetaInfo metaInfo,
> > > Map<String, String> map) {
> > >     public static PropertyProvider empty() {
> > >     public static PropertyProvider emptyMutable() {
> > >     public static PropertyProvider empty(MetaInfo metaInfo) {
> > >     public static PropertyProvider emptyMutable(MetaInfo metaInfo) {
> > >     public static PropertyProvider fromEnvironmentProperties() {
> > >     public static PropertyProvider fromSystemProperties() {
> > >     public static PropertyProvider freezed(PropertyProvider provider) {
> > >     public static PropertyProvider aggregate(AggregationPolicy
> > > mapping, MetaInfo metaInfo, PropertyProvider... providers){
> > >     public static PropertyProvider aggregate(PropertyProvider...
> > providers)
> > > {
> > >     public static PropertyProvider aggregate(List<PropertyProvider>
> > > providers) {
> > >     public static PropertyProvider aggregate(AggregationPolicy
> > > mapping, PropertyProvider... propertyMaps) {
> > >     public static PropertyProvider aggregate(AggregationPolicy
> > > mapping, List<PropertyProvider> providers) {
> > >     public static PropertyProvider mutable(PropertyProvider provider) {
> > >     public static PropertyProvider intersected(AggregationPolicy
> > > aggregationPolicy, PropertyProvider... providers) {
> > >     public static PropertyProvider intersected(PropertyProvider...
> > providers)
> > > {
> > >     public static PropertyProvider subtracted(PropertyProvider target,
> > > PropertyProvider... providers) {
> > >     public static PropertyProvider filtered(Predicate<String> filter,
> > > PropertyProvider provider) {
> > >     public static PropertyProvider
> > > contextual(Supplier<PropertyProvider> mapSupplier,
> > >                                               Supplier<String>
> > > isolationKeySupplier) {
> > >     public static PropertyProvider delegating(PropertyProvider
> > > mainMap, Map<String, String> parentMap) {
> > >     public static PropertyProvider replacing(PropertyProvider mainMap,
> > > Map<String, String> replacementMap) {
> > > }
> > >
> >
>

Re: UC: Combining PropertyProvider Instances

[Werner Keil <we...@gmail.com>]

While using XML a lot of it can be found that Multiconf did in Python.

(I am pretty sure most of Spring Config still uses plenty of XML, too, in
theory similar to Multiconf it could use a Groovy DSL, maybe if Pivotal
used Tamaya some day and the Groovy experts here considered such DSL a
worthy effort?;-D)




On Thu, Dec 4, 2014 at 2:53 PM, Tresch, Anatole <
anatole.tresch@credit-suisse.com> wrote:

> Hi Mark
>
> let me do a last try to clarifiy my ideas, I am open to discuss things
> (and do add some ideas at the end, so we can start the discussions right
> away ;) ):
>
> 1) my idea was to define a minimalistic interface for properties first. It
> should as simple as possible to implement. Just using Map<String,String> I
> found not optimal, since a Map requires several methods that are not easy
> to implement in all cases, such as size(). Also Spring provides such a
> similarl abstraction(PropertySource). For this I created PropertyProvider.
> 2) Looking at Configuration there is much more functionality needed to
> have a powerful config API, such as type support (getInt, getBoolean,
> getAdapted), registering of listeners, extension points (with, query). From
> our bank
>    additional methods were added for evaluation of the key ranges present,
> assuming hereby a.b.d: a is an area, containing an area b containing a key
> d. So I defined the Configuration interface.
> 3) Now there are very common types of properties sources you want to read
> from. I personally identified: classpath resources, files and URLs/URIs. So
> I added some kind of factory class (PropertyProviders), which allows to
> easily
>    construct them descriptively: PropertyProvider proc =
> PropertyProviders.fromPath("classpath:META-INF/cfg/config.ini");
> 4) In many companies you do not have a fixed configuration file name, you
> will have locations, where you consume all files (even recursively). So
> adding resource loader support similar to Spring was compelling:
>     PropertyProvider proc =
> PropertyProviders.fromPath("classpath:META-INF/cfg/**/*.ini");
> 5) Looking at what we have in Credit Suisse and what other prople told
> they have in their companies, simple priority based adding of maps is not
> enough. There are cases, where I want to filter out values, prevent
> overriding of certain values etc. Also different locations (classpath,
> files, remote) are mixed and depending on the source different priorities
> may be assigned. Finally configuration pairs themselves can be attributed
> (default entries, global default entries, explicit entries, ..), which also
> lead to different configuration priorities within the same source: or even
> config file, e.g:
>
> <config>
>   <defaults>
>    <a>aValue</a>
>   <b>bValue</b>
> </deault>
>
> <pluginConfig plugin="abc">
>   <a>aValue2</a>
> </pluginConfig>
>
> <server ip="1.2.4.5">
>      <b>bValue2</b>
> </server>
> </config>
>
> Since I saw definitively the most complex scenarios I wanted to support
> power users doing that kind of modelling more easily. One mechanism to
> achive this is allowing to aggregate and combine maps (=providers), so
> I added the combination methods on PropertyProviders like aggregate,
> intersect, filter, ...
> Now this seems more complicated for you guys than it seems for me ;) I can
> imagine different things that may be simplified:
>
> - Remove PropertyProvider/ PropertyProviders from the API part and just
> add corresponding functionality using a ConfigurationBuilder:
>
>   Configuration config =
> ConfigurationBuilder.create("myConfig").addPaths("ab/B/B",
> "../fhgg.xml").with(AggregationPolicy.OVERRIDE).addPaths("jhbjh/kjhkjh/").build();
>   Config aggregatedConfig = config.with(Aggregators.from(config);
>
> - We still could put the PropertyProvider to the core implementation
> module, or even add it as a separate module, for the ones that like a more
> leightweight implementation SPI ;)
>
> Configuration conf = Configuration.current(); will still be the accessor
> method of choice. Though I am well aware of the fact that the abstractions
> (SPIs) required to evaluate the right configuration during runtime  is not
> yet discussed. Also the current state in the repo in that area is far away
> from finished (I currently try to cleanup a few things, especially
> artifacts that are duplicates...).
>
> Best,
> Anatole
>
>
>
> -----Original Message-----
> From: Mark Struberg [mailto:struberg@yahoo.de]
> Sent: Donnerstag, 4. Dezember 2014 11:08
> To: dev@tamaya.incubator.apache.org
> Subject: Re: UC: Combining PropertyProvider Instances
>
> I think I need a bit time to understand this. I've tried to grasp it for 2
> days now and this might mean that it is too complicated if not even I get
> it. Why are we doing this so complicated? Pulling in all the database logic
> into a tool which main target is that it should be really easy to use?
> What benefit does this add over hiding those details? e.g. in DeltaSpike I
> would just add a DatabaseConfigSource and be done. The user would not even
> see that it magically picks up the values from the db then.
>
>
> LieGrue,
> strub
>
>
>
>
>
> > On Tuesday, 2 December 2014, 14:02, Anatole Tresch <an...@apache.org>
> wrote:
> > >    Hi all
> > find following an additional part of use cases for comment. Note that
> this
> > area IMO definitively may be improved in several ways. Always keep in
> mind
> > that I try to setup the design guide similarly in parallel. It can be
> > easily accessed from the GH mirror:
> >
> >
> https://github.com/apache/incubator-tamaya/blob/master/docs/design/2_CoreConcepts.adoc
> >
> > Cheers,
> > Anatole
> >
> > Combining Property Providers
> >
> > Looking at the structures of configuration system used by large companies
> > we typically encounter some kind of configuration hierarchies that are
> > combined in arbitrary ways. Users of the systems are typically not aware
> of
> > the complexities in this area, since they simply know the possible
> > locations, formats and the overriding policies. Framework providers on
> the
> > other side must face the complexities and it would be very useful if
> Tamaya
> > can support here by providing prebuilt functionality that helps
> > implementing these aspects. All this leads to the feature set of
> combining
> > property providers. Hereby the following strategies are useful:
> >
> >    -
> >
> >    aggregating providers, hereby later providers added
> >     -
> >
> >       override any existing entries from earlier providers
> >        -
> >
> >       combine conflicting entries from earlier providers, e.g. into a
> >       comma-separated structure.
> >        -
> >
> >       may throw a ConfigExcepotion ig entries are conflicting
> >        -
> >
> >       may only add entries not yet defined by former providers,
> preventing
> >       entries that are already present to be overwritte
> >        -
> >
> >       any custom aggregation strategy, which may be a mix of above
> >         -
> >
> >    intersecting providers
> >     -
> >
> >    subtracting providers
> >     -
> >
> >    filtering providers
> >
> >   These common functionality is provided by the PropertyProviders
> > singleton. Additionally to the base strategies above a MetaInfo instance
> > can be passed optionally as well to define the meta information for the
> > newly created provider instances. Let’s assume we have two property
> > providers with the following data:
> >   Provider 1
> >
> > a=a
> > b=b
> > c=c
> > g=g
> > h=h
> > i=i
> >
> >   Provider 2
> >
> > a=A
> > b=B
> > c=C
> > d=D
> > e=E
> > f=F
> >
> >   Looking in detail you see that the entries a,b,c are present in both
> > providers, whereas d,e,f are only present in provider 1, and g,h,i only
> in
> > provider 2.
> >   Example Combining PropertyProviders
> >
> > PropertyProvider provider1 = ...
> > PropertyProvider provider2 = ...
> > // aggregate, hereby values from provider 2 override values from
> provider 1
> > PropertyProvider unionOverriding =
> > PropertyProviders.aggregate(AggregationPolicy.OVERRIDE(), provider1,
> > provider2);*System*.out.println("unionOverriding: " +
> > unionOverriding);
> > // ignore duplicates, values present in provider 1 are not overriden
> > by provider 2
> > PropertyProvider unionIgnoringDuplicates =
> > PropertyProviders.aggregate(AggregationPolicy.IGNORE_DUPLICATES(),
> > provider1, provider2);*System*.out.println("unionIgnoringDuplicates: "
> > + unionIgnoringDuplicates);
> > // this variant combines/maps duplicate values into a new value
> > PropertyProvider unionCombined =
> > PropertyProviders.aggregate(AggregationPolicy.COMBINE(), provider1,
> > provider2);*System*.out.println("unionCombined: " + unionCombined);
> > // This variant throws an exception since there are key/value paris in
> > both providers, but with different values*try*{
> >     PropertyProviders.aggregate(AggregationPolicy.EXCEPTION(),
> > provider1, provider2);
> > }*catch*(ConfigException e){
> >     // expected!
> > }
> >
> >   The example above produces the following outpout:
> >   Example Combining PropertyProviders
> >
> > AggregatedPropertyProvider{
> >   (name = dynamicAggregationTests)
> >   a = "[a][A]"
> >   b = "[b][B]"
> >   c = "[c][C]"
> >   d = "[D]"
> >   e = "[E]"
> >   f = "[F]"
> >   g = "[g]"
> >   h = "[h]"
> >   i = "[i]"
> > }
> > unionOverriding: AggregatedPropertyProvider{
> >   (name = <noname>)
> >   a = "A"
> >   b = "B"
> >   c = "C"
> >   d = "D"
> >   e = "E"
> >   f = "F"
> >   g = "g"
> >   h = "h"
> >   i = "i"
> > }
> > unionIgnoringDuplicates: AggregatedPropertyProvider{
> >   (name = <noname>)
> >   a = "a"
> >   b = "b"
> >   c = "c"
> >   d = "D"
> >   e = "E"
> >   f = "F"
> >   g = "g"
> >   h = "h"
> >   i = "i"
> > }
> > unionCombined: AggregatedPropertyProvider{
> >   (name = <noname>)
> >   a = "a,A"
> >   b = "b,B"
> >   c = "c,C"
> >   d = "D"
> >   e = "E"
> >   f = "F"
> >   g = "g"
> >   h = "h"
> >   i = "i"
> > }
> >
> >   No AggregationPolicy is also an interface that can be implemented:
> >   AggregationPolicy Interface
> >
> > @FunctionalInterface*public* *interface* *AggregationPolicy* {
> >     *String* aggregate(*String* key, *String* value1, *String* value2);
> > }
> >
> >   So we can also define our own aggregation strategy using a Lambda
> > expression:
> >   Use a Custom AggregationPolicy
> >
> > PropertyProvider provider1 = ...;
> > PropertyProvider provider2 = ...;
> > PropertyProvider props = PropertyProviders.aggregate(
> >       (k, v1, v2) -> (v1 != null ? v1 : "") + '[' + v2 +
> > "]",
> >       MetaInfo.of("dynamicAggregationTests"),
> >       props1, props2);*System*.out.println(props);
> >
> >   Additionally we also pass here an instance of MetaInfo. The output of
> > this code snippet is as follows:
> >   Listing of dynamic aggregation policy
> >
> > AggregatedPropertyProvider{
> >   (name = dynamicAggregationTests)
> >   a = "[a][A]"
> >   b = "[b][B]"
> >   c = "[c][C]"
> >   d = "[D]"
> >   e = "[E]"
> >   f = "[F]"
> >   g = "[g]"
> >   h = "[h]"
> >   i = "[i]"
> > }
> >
> >   Summarizing the PropertyProviders singleton allows to combine providers
> > in various forms:
> >   Methods provided on PropertyProviders
> >
> > public final class PropertyProviders {
> >
> >     private PropertyProviders() {}
> >
> >     public static PropertyProvider fromArgs(String... args) {
> >     public static PropertyProvider fromArgs(MetaInfo metaInfo, String...
> args)
> > {
> >     public static PropertyProvider fromPaths(AggregationPolicy
> > aggregationPolicy, String... paths) {
> >     public static PropertyProvider fromPaths(String... paths) {
> >     public static PropertyProvider fromPaths(List<String> paths) {
> >     public static PropertyProvider fromPaths(AggregationPolicy
> > aggregationPolicy, List<String> paths) {
> >     public static PropertyProvider fromPaths(MetaInfo metaInfo,
> > List<String> paths) {
> >     public static PropertyProvider fromPaths(AggregationPolicy
> > aggregationPolicy, MetaInfo metaInfo, List<String> paths) {
> >     public static PropertyProvider fromUris(URI... uris) {
> >     public static PropertyProvider fromUris(AggregationPolicy
> > aggregationPolicy, URI... uris) {
> >     public static PropertyProvider fromUris(List<URI> uris) {
> >     public static PropertyProvider fromUris(AggregationPolicy
> > aggregationPolicy, List<URI> uris) {
> >     public static PropertyProvider fromUris(MetaInfo metaInfo, URI...
> uris)
> > {
> >     public static PropertyProvider fromUris(AggregationPolicy
> > aggregationPolicy, MetaInfo metaInfo, URI... uris) {
> >     public static PropertyProvider fromUris(MetaInfo metaInfo, List<URI>
> > uris) {
> >     public static PropertyProvider fromUris(AggregationPolicy
> > aggregationPolicy, MetaInfo metaInfo, List<URI> uris) {
> >     public static PropertyProvider fromMap(Map<String, String> map) {
> >     public static PropertyProvider fromMap(MetaInfo metaInfo,
> > Map<String, String> map) {
> >     public static PropertyProvider empty() {
> >     public static PropertyProvider emptyMutable() {
> >     public static PropertyProvider empty(MetaInfo metaInfo) {
> >     public static PropertyProvider emptyMutable(MetaInfo metaInfo) {
> >     public static PropertyProvider fromEnvironmentProperties() {
> >     public static PropertyProvider fromSystemProperties() {
> >     public static PropertyProvider freezed(PropertyProvider provider) {
> >     public static PropertyProvider aggregate(AggregationPolicy
> > mapping, MetaInfo metaInfo, PropertyProvider... providers){
> >     public static PropertyProvider aggregate(PropertyProvider...
> providers)
> > {
> >     public static PropertyProvider aggregate(List<PropertyProvider>
> > providers) {
> >     public static PropertyProvider aggregate(AggregationPolicy
> > mapping, PropertyProvider... propertyMaps) {
> >     public static PropertyProvider aggregate(AggregationPolicy
> > mapping, List<PropertyProvider> providers) {
> >     public static PropertyProvider mutable(PropertyProvider provider) {
> >     public static PropertyProvider intersected(AggregationPolicy
> > aggregationPolicy, PropertyProvider... providers) {
> >     public static PropertyProvider intersected(PropertyProvider...
> providers)
> > {
> >     public static PropertyProvider subtracted(PropertyProvider target,
> > PropertyProvider... providers) {
> >     public static PropertyProvider filtered(Predicate<String> filter,
> > PropertyProvider provider) {
> >     public static PropertyProvider
> > contextual(Supplier<PropertyProvider> mapSupplier,
> >                                               Supplier<String>
> > isolationKeySupplier) {
> >     public static PropertyProvider delegating(PropertyProvider
> > mainMap, Map<String, String> parentMap) {
> >     public static PropertyProvider replacing(PropertyProvider mainMap,
> > Map<String, String> replacementMap) {
> > }
> >
>

RE: UC: Combining PropertyProvider Instances

["Tresch, Anatole " <an...@credit-suisse.com>]

Hi Mark

let me do a last try to clarifiy my ideas, I am open to discuss things (and do add some ideas at the end, so we can start the discussions right away ;) ):

1) my idea was to define a minimalistic interface for properties first. It should as simple as possible to implement. Just using Map<String,String> I found not optimal, since a Map requires several methods that are not easy to implement in all cases, such as size(). Also Spring provides such a similarl abstraction(PropertySource). For this I created PropertyProvider.
2) Looking at Configuration there is much more functionality needed to have a powerful config API, such as type support (getInt, getBoolean, getAdapted), registering of listeners, extension points (with, query). From our bank
   additional methods were added for evaluation of the key ranges present, assuming hereby a.b.d: a is an area, containing an area b containing a key d. So I defined the Configuration interface.
3) Now there are very common types of properties sources you want to read from. I personally identified: classpath resources, files and URLs/URIs. So I added some kind of factory class (PropertyProviders), which allows to easily
   construct them descriptively: PropertyProvider proc = PropertyProviders.fromPath("classpath:META-INF/cfg/config.ini");
4) In many companies you do not have a fixed configuration file name, you will have locations, where you consume all files (even recursively). So adding resource loader support similar to Spring was compelling:
    PropertyProvider proc = PropertyProviders.fromPath("classpath:META-INF/cfg/**/*.ini");
5) Looking at what we have in Credit Suisse and what other prople told they have in their companies, simple priority based adding of maps is not enough. There are cases, where I want to filter out values, prevent  overriding of certain values etc. Also different locations (classpath, files, remote) are mixed and depending on the source different priorities may be assigned. Finally configuration pairs themselves can be attributed (default entries, global default entries, explicit entries, ..), which also lead to different configuration priorities within the same source: or even config file, e.g:

<config>
  <defaults>
   <a>aValue</a>
  <b>bValue</b>
</deault>

<pluginConfig plugin="abc">
  <a>aValue2</a>
</pluginConfig>

<server ip="1.2.4.5">
     <b>bValue2</b>
</server>
</config>

Since I saw definitively the most complex scenarios I wanted to support power users doing that kind of modelling more easily. One mechanism to achive this is allowing to aggregate and combine maps (=providers), so
I added the combination methods on PropertyProviders like aggregate, intersect, filter, ...
Now this seems more complicated for you guys than it seems for me ;) I can imagine different things that may be simplified:

- Remove PropertyProvider/ PropertyProviders from the API part and just add corresponding functionality using a ConfigurationBuilder:

  Configuration config =  ConfigurationBuilder.create("myConfig").addPaths("ab/B/B", "../fhgg.xml").with(AggregationPolicy.OVERRIDE).addPaths("jhbjh/kjhkjh/").build();
  Config aggregatedConfig = config.with(Aggregators.from(config);

- We still could put the PropertyProvider to the core implementation module, or even add it as a separate module, for the ones that like a more leightweight implementation SPI ;)

Configuration conf = Configuration.current(); will still be the accessor method of choice. Though I am well aware of the fact that the abstractions (SPIs) required to evaluate the right configuration during runtime  is not yet discussed. Also the current state in the repo in that area is far away from finished (I currently try to cleanup a few things, especially artifacts that are duplicates...).

Best,
Anatole



-----Original Message-----
From: Mark Struberg [mailto:struberg@yahoo.de] 
Sent: Donnerstag, 4. Dezember 2014 11:08
To: dev@tamaya.incubator.apache.org
Subject: Re: UC: Combining PropertyProvider Instances

I think I need a bit time to understand this. I've tried to grasp it for 2 days now and this might mean that it is too complicated if not even I get it. Why are we doing this so complicated? Pulling in all the database logic into a tool which main target is that it should be really easy to use?
What benefit does this add over hiding those details? e.g. in DeltaSpike I would just add a DatabaseConfigSource and be done. The user would not even see that it magically picks up the values from the db then. 


LieGrue,
strub





> On Tuesday, 2 December 2014, 14:02, Anatole Tresch <an...@apache.org> wrote:
> >    Hi all
> find following an additional part of use cases for comment. Note that this
> area IMO definitively may be improved in several ways. Always keep in mind
> that I try to setup the design guide similarly in parallel. It can be
> easily accessed from the GH mirror:
> 
> https://github.com/apache/incubator-tamaya/blob/master/docs/design/2_CoreConcepts.adoc
> 
> Cheers,
> Anatole
> 
> Combining Property Providers
> 
> Looking at the structures of configuration system used by large companies
> we typically encounter some kind of configuration hierarchies that are
> combined in arbitrary ways. Users of the systems are typically not aware of
> the complexities in this area, since they simply know the possible
> locations, formats and the overriding policies. Framework providers on the
> other side must face the complexities and it would be very useful if Tamaya
> can support here by providing prebuilt functionality that helps
> implementing these aspects. All this leads to the feature set of combining
> property providers. Hereby the following strategies are useful:
> 
>    -
> 
>    aggregating providers, hereby later providers added
>     -
> 
>       override any existing entries from earlier providers
>        -
> 
>       combine conflicting entries from earlier providers, e.g. into a
>       comma-separated structure.
>        -
> 
>       may throw a ConfigExcepotion ig entries are conflicting
>        -
> 
>       may only add entries not yet defined by former providers, preventing
>       entries that are already present to be overwritte
>        -
> 
>       any custom aggregation strategy, which may be a mix of above
>         -
> 
>    intersecting providers
>     -
> 
>    subtracting providers
>     -
> 
>    filtering providers
> 
>   These common functionality is provided by the PropertyProviders
> singleton. Additionally to the base strategies above a MetaInfo instance
> can be passed optionally as well to define the meta information for the
> newly created provider instances. Let’s assume we have two property
> providers with the following data:
>   Provider 1
> 
> a=a
> b=b
> c=c
> g=g
> h=h
> i=i
> 
>   Provider 2
> 
> a=A
> b=B
> c=C
> d=D
> e=E
> f=F
> 
>   Looking in detail you see that the entries a,b,c are present in both
> providers, whereas d,e,f are only present in provider 1, and g,h,i only in
> provider 2.
>   Example Combining PropertyProviders
> 
> PropertyProvider provider1 = ...
> PropertyProvider provider2 = ...
> // aggregate, hereby values from provider 2 override values from provider 1
> PropertyProvider unionOverriding =
> PropertyProviders.aggregate(AggregationPolicy.OVERRIDE(), provider1,
> provider2);*System*.out.println("unionOverriding: " +
> unionOverriding);
> // ignore duplicates, values present in provider 1 are not overriden
> by provider 2
> PropertyProvider unionIgnoringDuplicates =
> PropertyProviders.aggregate(AggregationPolicy.IGNORE_DUPLICATES(),
> provider1, provider2);*System*.out.println("unionIgnoringDuplicates: "
> + unionIgnoringDuplicates);
> // this variant combines/maps duplicate values into a new value
> PropertyProvider unionCombined =
> PropertyProviders.aggregate(AggregationPolicy.COMBINE(), provider1,
> provider2);*System*.out.println("unionCombined: " + unionCombined);
> // This variant throws an exception since there are key/value paris in
> both providers, but with different values*try*{
>     PropertyProviders.aggregate(AggregationPolicy.EXCEPTION(),
> provider1, provider2);
> }*catch*(ConfigException e){
>     // expected!
> }
> 
>   The example above produces the following outpout:
>   Example Combining PropertyProviders
> 
> AggregatedPropertyProvider{
>   (name = dynamicAggregationTests)
>   a = "[a][A]"
>   b = "[b][B]"
>   c = "[c][C]"
>   d = "[D]"
>   e = "[E]"
>   f = "[F]"
>   g = "[g]"
>   h = "[h]"
>   i = "[i]"
> }
> unionOverriding: AggregatedPropertyProvider{
>   (name = <noname>)
>   a = "A"
>   b = "B"
>   c = "C"
>   d = "D"
>   e = "E"
>   f = "F"
>   g = "g"
>   h = "h"
>   i = "i"
> }
> unionIgnoringDuplicates: AggregatedPropertyProvider{
>   (name = <noname>)
>   a = "a"
>   b = "b"
>   c = "c"
>   d = "D"
>   e = "E"
>   f = "F"
>   g = "g"
>   h = "h"
>   i = "i"
> }
> unionCombined: AggregatedPropertyProvider{
>   (name = <noname>)
>   a = "a,A"
>   b = "b,B"
>   c = "c,C"
>   d = "D"
>   e = "E"
>   f = "F"
>   g = "g"
>   h = "h"
>   i = "i"
> }
> 
>   No AggregationPolicy is also an interface that can be implemented:
>   AggregationPolicy Interface
> 
> @FunctionalInterface*public* *interface* *AggregationPolicy* {
>     *String* aggregate(*String* key, *String* value1, *String* value2);
> }
> 
>   So we can also define our own aggregation strategy using a Lambda
> expression:
>   Use a Custom AggregationPolicy
> 
> PropertyProvider provider1 = ...;
> PropertyProvider provider2 = ...;
> PropertyProvider props = PropertyProviders.aggregate(
>       (k, v1, v2) -> (v1 != null ? v1 : "") + '[' + v2 + 
> "]",
>       MetaInfo.of("dynamicAggregationTests"),
>       props1, props2);*System*.out.println(props);
> 
>   Additionally we also pass here an instance of MetaInfo. The output of
> this code snippet is as follows:
>   Listing of dynamic aggregation policy
> 
> AggregatedPropertyProvider{
>   (name = dynamicAggregationTests)
>   a = "[a][A]"
>   b = "[b][B]"
>   c = "[c][C]"
>   d = "[D]"
>   e = "[E]"
>   f = "[F]"
>   g = "[g]"
>   h = "[h]"
>   i = "[i]"
> }
> 
>   Summarizing the PropertyProviders singleton allows to combine providers
> in various forms:
>   Methods provided on PropertyProviders
> 
> public final class PropertyProviders {
> 
>     private PropertyProviders() {}
> 
>     public static PropertyProvider fromArgs(String... args) {
>     public static PropertyProvider fromArgs(MetaInfo metaInfo, String... args) 
> {
>     public static PropertyProvider fromPaths(AggregationPolicy
> aggregationPolicy, String... paths) {
>     public static PropertyProvider fromPaths(String... paths) {
>     public static PropertyProvider fromPaths(List<String> paths) {
>     public static PropertyProvider fromPaths(AggregationPolicy
> aggregationPolicy, List<String> paths) {
>     public static PropertyProvider fromPaths(MetaInfo metaInfo,
> List<String> paths) {
>     public static PropertyProvider fromPaths(AggregationPolicy
> aggregationPolicy, MetaInfo metaInfo, List<String> paths) {
>     public static PropertyProvider fromUris(URI... uris) {
>     public static PropertyProvider fromUris(AggregationPolicy
> aggregationPolicy, URI... uris) {
>     public static PropertyProvider fromUris(List<URI> uris) {
>     public static PropertyProvider fromUris(AggregationPolicy
> aggregationPolicy, List<URI> uris) {
>     public static PropertyProvider fromUris(MetaInfo metaInfo, URI... uris) 
> {
>     public static PropertyProvider fromUris(AggregationPolicy
> aggregationPolicy, MetaInfo metaInfo, URI... uris) {
>     public static PropertyProvider fromUris(MetaInfo metaInfo, List<URI> 
> uris) {
>     public static PropertyProvider fromUris(AggregationPolicy
> aggregationPolicy, MetaInfo metaInfo, List<URI> uris) {
>     public static PropertyProvider fromMap(Map<String, String> map) {
>     public static PropertyProvider fromMap(MetaInfo metaInfo,
> Map<String, String> map) {
>     public static PropertyProvider empty() {
>     public static PropertyProvider emptyMutable() {
>     public static PropertyProvider empty(MetaInfo metaInfo) {
>     public static PropertyProvider emptyMutable(MetaInfo metaInfo) {
>     public static PropertyProvider fromEnvironmentProperties() {
>     public static PropertyProvider fromSystemProperties() {
>     public static PropertyProvider freezed(PropertyProvider provider) {
>     public static PropertyProvider aggregate(AggregationPolicy
> mapping, MetaInfo metaInfo, PropertyProvider... providers){
>     public static PropertyProvider aggregate(PropertyProvider... providers) 
> {
>     public static PropertyProvider aggregate(List<PropertyProvider> 
> providers) {
>     public static PropertyProvider aggregate(AggregationPolicy
> mapping, PropertyProvider... propertyMaps) {
>     public static PropertyProvider aggregate(AggregationPolicy
> mapping, List<PropertyProvider> providers) {
>     public static PropertyProvider mutable(PropertyProvider provider) {
>     public static PropertyProvider intersected(AggregationPolicy
> aggregationPolicy, PropertyProvider... providers) {
>     public static PropertyProvider intersected(PropertyProvider... providers) 
> {
>     public static PropertyProvider subtracted(PropertyProvider target,
> PropertyProvider... providers) {
>     public static PropertyProvider filtered(Predicate<String> filter,
> PropertyProvider provider) {
>     public static PropertyProvider
> contextual(Supplier<PropertyProvider> mapSupplier,
>                                               Supplier<String>
> isolationKeySupplier) {
>     public static PropertyProvider delegating(PropertyProvider
> mainMap, Map<String, String> parentMap) {
>     public static PropertyProvider replacing(PropertyProvider mainMap,
> Map<String, String> replacementMap) {
> }
>

Re: UC: Combining PropertyProvider Instances

["Oliver B. Fischer" <o....@swe-blog.net>]

+1
Am 04.12.14 um 11:07 schrieb Mark Struberg:
> I think I need a bit time to understand this. I've tried to grasp it for 2 days now and this might mean that it is too complicated if not even I get it. Why are we doing this so complicated? Pulling in all the database logic into a tool which main target is that it should be really easy to use?
> What benefit does this add over hiding those details? e.g. in DeltaSpike I would just add a DatabaseConfigSource and be done. The user would not even see that it magically picks up the values from the db then.
>
>
> LieGrue,
> strub
>
>
>
>
>
>> On Tuesday, 2 December 2014, 14:02, Anatole Tresch <an...@apache.org> wrote:
>> find following an additional part of use cases for comment. Note that this
>> area IMO definitively may be improved in several ways. Always keep in mind
>> that I try to setup the design guide similarly in parallel. It can be
>> easily accessed from the GH mirror:
>>
>> https://github.com/apache/incubator-tamaya/blob/master/docs/design/2_CoreConcepts.adoc
>>
>> Cheers,
>> Anatole
>>
>> Combining Property Providers
>>
>> Looking at the structures of configuration system used by large companies
>> we typically encounter some kind of configuration hierarchies that are
>> combined in arbitrary ways. Users of the systems are typically not aware of
>> the complexities in this area, since they simply know the possible
>> locations, formats and the overriding policies. Framework providers on the
>> other side must face the complexities and it would be very useful if Tamaya
>> can support here by providing prebuilt functionality that helps
>> implementing these aspects. All this leads to the feature set of combining
>> property providers. Hereby the following strategies are useful:
>>
>>     -
>>
>>     aggregating providers, hereby later providers added
>>      -
>>
>>        override any existing entries from earlier providers
>>         -
>>
>>        combine conflicting entries from earlier providers, e.g. into a
>>        comma-separated structure.
>>         -
>>
>>        may throw a ConfigExcepotion ig entries are conflicting
>>         -
>>
>>        may only add entries not yet defined by former providers, preventing
>>        entries that are already present to be overwritte
>>         -
>>
>>        any custom aggregation strategy, which may be a mix of above
>>          -
>>
>>     intersecting providers
>>      -
>>
>>     subtracting providers
>>      -
>>
>>     filtering providers
>>
>>    These common functionality is provided by the PropertyProviders
>> singleton. Additionally to the base strategies above a MetaInfo instance
>> can be passed optionally as well to define the meta information for the
>> newly created provider instances. Let’s assume we have two property
>> providers with the following data:
>>    Provider 1
>>
>> a=a
>> b=b
>> c=c
>> g=g
>> h=h
>> i=i
>>
>>    Provider 2
>>
>> a=A
>> b=B
>> c=C
>> d=D
>> e=E
>> f=F
>>
>>    Looking in detail you see that the entries a,b,c are present in both
>> providers, whereas d,e,f are only present in provider 1, and g,h,i only in
>> provider 2.
>>    Example Combining PropertyProviders
>>
>> PropertyProvider provider1 = ...
>> PropertyProvider provider2 = ...
>> // aggregate, hereby values from provider 2 override values from provider 1
>> PropertyProvider unionOverriding =
>> PropertyProviders.aggregate(AggregationPolicy.OVERRIDE(), provider1,
>> provider2);*System*.out.println("unionOverriding: " +
>> unionOverriding);
>> // ignore duplicates, values present in provider 1 are not overriden
>> by provider 2
>> PropertyProvider unionIgnoringDuplicates =
>> PropertyProviders.aggregate(AggregationPolicy.IGNORE_DUPLICATES(),
>> provider1, provider2);*System*.out.println("unionIgnoringDuplicates: "
>> + unionIgnoringDuplicates);
>> // this variant combines/maps duplicate values into a new value
>> PropertyProvider unionCombined =
>> PropertyProviders.aggregate(AggregationPolicy.COMBINE(), provider1,
>> provider2);*System*.out.println("unionCombined: " + unionCombined);
>> // This variant throws an exception since there are key/value paris in
>> both providers, but with different values*try*{
>>      PropertyProviders.aggregate(AggregationPolicy.EXCEPTION(),
>> provider1, provider2);
>> }*catch*(ConfigException e){
>>      // expected!
>> }
>>
>>    The example above produces the following outpout:
>>    Example Combining PropertyProviders
>>
>> AggregatedPropertyProvider{
>>    (name = dynamicAggregationTests)
>>    a = "[a][A]"
>>    b = "[b][B]"
>>    c = "[c][C]"
>>    d = "[D]"
>>    e = "[E]"
>>    f = "[F]"
>>    g = "[g]"
>>    h = "[h]"
>>    i = "[i]"
>> }
>> unionOverriding: AggregatedPropertyProvider{
>>    (name = <noname>)
>>    a = "A"
>>    b = "B"
>>    c = "C"
>>    d = "D"
>>    e = "E"
>>    f = "F"
>>    g = "g"
>>    h = "h"
>>    i = "i"
>> }
>> unionIgnoringDuplicates: AggregatedPropertyProvider{
>>    (name = <noname>)
>>    a = "a"
>>    b = "b"
>>    c = "c"
>>    d = "D"
>>    e = "E"
>>    f = "F"
>>    g = "g"
>>    h = "h"
>>    i = "i"
>> }
>> unionCombined: AggregatedPropertyProvider{
>>    (name = <noname>)
>>    a = "a,A"
>>    b = "b,B"
>>    c = "c,C"
>>    d = "D"
>>    e = "E"
>>    f = "F"
>>    g = "g"
>>    h = "h"
>>    i = "i"
>> }
>>
>>    No AggregationPolicy is also an interface that can be implemented:
>>    AggregationPolicy Interface
>>
>> @FunctionalInterface*public* *interface* *AggregationPolicy* {
>>      *String* aggregate(*String* key, *String* value1, *String* value2);
>> }
>>
>>    So we can also define our own aggregation strategy using a Lambda
>> expression:
>>    Use a Custom AggregationPolicy
>>
>> PropertyProvider provider1 = ...;
>> PropertyProvider provider2 = ...;
>> PropertyProvider props = PropertyProviders.aggregate(
>>        (k, v1, v2) -> (v1 != null ? v1 : "") + '[' + v2 +
>> "]",
>>        MetaInfo.of("dynamicAggregationTests"),
>>        props1, props2);*System*.out.println(props);
>>
>>    Additionally we also pass here an instance of MetaInfo. The output of
>> this code snippet is as follows:
>>    Listing of dynamic aggregation policy
>>
>> AggregatedPropertyProvider{
>>    (name = dynamicAggregationTests)
>>    a = "[a][A]"
>>    b = "[b][B]"
>>    c = "[c][C]"
>>    d = "[D]"
>>    e = "[E]"
>>    f = "[F]"
>>    g = "[g]"
>>    h = "[h]"
>>    i = "[i]"
>> }
>>
>>    Summarizing the PropertyProviders singleton allows to combine providers
>> in various forms:
>>    Methods provided on PropertyProviders
>>
>> public final class PropertyProviders {
>>
>>      private PropertyProviders() {}
>>
>>      public static PropertyProvider fromArgs(String... args) {
>>      public static PropertyProvider fromArgs(MetaInfo metaInfo, String... args)
>> {
>>      public static PropertyProvider fromPaths(AggregationPolicy
>> aggregationPolicy, String... paths) {
>>      public static PropertyProvider fromPaths(String... paths) {
>>      public static PropertyProvider fromPaths(List<String> paths) {
>>      public static PropertyProvider fromPaths(AggregationPolicy
>> aggregationPolicy, List<String> paths) {
>>      public static PropertyProvider fromPaths(MetaInfo metaInfo,
>> List<String> paths) {
>>      public static PropertyProvider fromPaths(AggregationPolicy
>> aggregationPolicy, MetaInfo metaInfo, List<String> paths) {
>>      public static PropertyProvider fromUris(URI... uris) {
>>      public static PropertyProvider fromUris(AggregationPolicy
>> aggregationPolicy, URI... uris) {
>>      public static PropertyProvider fromUris(List<URI> uris) {
>>      public static PropertyProvider fromUris(AggregationPolicy
>> aggregationPolicy, List<URI> uris) {
>>      public static PropertyProvider fromUris(MetaInfo metaInfo, URI... uris)
>> {
>>      public static PropertyProvider fromUris(AggregationPolicy
>> aggregationPolicy, MetaInfo metaInfo, URI... uris) {
>>      public static PropertyProvider fromUris(MetaInfo metaInfo, List<URI>
>> uris) {
>>      public static PropertyProvider fromUris(AggregationPolicy
>> aggregationPolicy, MetaInfo metaInfo, List<URI> uris) {
>>      public static PropertyProvider fromMap(Map<String, String> map) {
>>      public static PropertyProvider fromMap(MetaInfo metaInfo,
>> Map<String, String> map) {
>>      public static PropertyProvider empty() {
>>      public static PropertyProvider emptyMutable() {
>>      public static PropertyProvider empty(MetaInfo metaInfo) {
>>      public static PropertyProvider emptyMutable(MetaInfo metaInfo) {
>>      public static PropertyProvider fromEnvironmentProperties() {
>>      public static PropertyProvider fromSystemProperties() {
>>      public static PropertyProvider freezed(PropertyProvider provider) {
>>      public static PropertyProvider aggregate(AggregationPolicy
>> mapping, MetaInfo metaInfo, PropertyProvider... providers){
>>      public static PropertyProvider aggregate(PropertyProvider... providers)
>> {
>>      public static PropertyProvider aggregate(List<PropertyProvider>
>> providers) {
>>      public static PropertyProvider aggregate(AggregationPolicy
>> mapping, PropertyProvider... propertyMaps) {
>>      public static PropertyProvider aggregate(AggregationPolicy
>> mapping, List<PropertyProvider> providers) {
>>      public static PropertyProvider mutable(PropertyProvider provider) {
>>      public static PropertyProvider intersected(AggregationPolicy
>> aggregationPolicy, PropertyProvider... providers) {
>>      public static PropertyProvider intersected(PropertyProvider... providers)
>> {
>>      public static PropertyProvider subtracted(PropertyProvider target,
>> PropertyProvider... providers) {
>>      public static PropertyProvider filtered(Predicate<String> filter,
>> PropertyProvider provider) {
>>      public static PropertyProvider
>> contextual(Supplier<PropertyProvider> mapSupplier,
>>                                                Supplier<String>
>> isolationKeySupplier) {
>>      public static PropertyProvider delegating(PropertyProvider
>> mainMap, Map<String, String> parentMap) {
>>      public static PropertyProvider replacing(PropertyProvider mainMap,
>> Map<String, String> replacementMap) {
>> }
>>

-- 
N Oliver B. Fischer
A Schönhauser Allee 64, 10437 Berlin, Deutschland/Germany
P +49 30 44793251
M +49 178 7903538
E o.b.fischer@swe-blog.net
S oliver.b.fischer
J oliver.b.fischer@jabber.org
X http://xing.to/obf

Re: UC: Combining PropertyProvider Instances

[Gerhard Petracek <ge...@gmail.com>]

@mark: +1

regards,
gerhard



2014-12-04 11:07 GMT+01:00 Mark Struberg <st...@yahoo.de>:

> I think I need a bit time to understand this. I've tried to grasp it for 2
> days now and this might mean that it is too complicated if not even I get
> it. Why are we doing this so complicated? Pulling in all the database logic
> into a tool which main target is that it should be really easy to use?
> What benefit does this add over hiding those details? e.g. in DeltaSpike I
> would just add a DatabaseConfigSource and be done. The user would not even
> see that it magically picks up the values from the db then.
>
>
> LieGrue,
> strub
>
>
>
>
>
> > On Tuesday, 2 December 2014, 14:02, Anatole Tresch <an...@apache.org>
> wrote:
> > >    Hi all
> > find following an additional part of use cases for comment. Note that
> this
> > area IMO definitively may be improved in several ways. Always keep in
> mind
> > that I try to setup the design guide similarly in parallel. It can be
> > easily accessed from the GH mirror:
> >
> >
> https://github.com/apache/incubator-tamaya/blob/master/docs/design/2_CoreConcepts.adoc
> >
> > Cheers,
> > Anatole
> >
> > Combining Property Providers
> >
> > Looking at the structures of configuration system used by large companies
> > we typically encounter some kind of configuration hierarchies that are
> > combined in arbitrary ways. Users of the systems are typically not aware
> of
> > the complexities in this area, since they simply know the possible
> > locations, formats and the overriding policies. Framework providers on
> the
> > other side must face the complexities and it would be very useful if
> Tamaya
> > can support here by providing prebuilt functionality that helps
> > implementing these aspects. All this leads to the feature set of
> combining
> > property providers. Hereby the following strategies are useful:
> >
> >    -
> >
> >    aggregating providers, hereby later providers added
> >     -
> >
> >       override any existing entries from earlier providers
> >        -
> >
> >       combine conflicting entries from earlier providers, e.g. into a
> >       comma-separated structure.
> >        -
> >
> >       may throw a ConfigExcepotion ig entries are conflicting
> >        -
> >
> >       may only add entries not yet defined by former providers,
> preventing
> >       entries that are already present to be overwritte
> >        -
> >
> >       any custom aggregation strategy, which may be a mix of above
> >         -
> >
> >    intersecting providers
> >     -
> >
> >    subtracting providers
> >     -
> >
> >    filtering providers
> >
> >   These common functionality is provided by the PropertyProviders
> > singleton. Additionally to the base strategies above a MetaInfo instance
> > can be passed optionally as well to define the meta information for the
> > newly created provider instances. Let’s assume we have two property
> > providers with the following data:
> >   Provider 1
> >
> > a=a
> > b=b
> > c=c
> > g=g
> > h=h
> > i=i
> >
> >   Provider 2
> >
> > a=A
> > b=B
> > c=C
> > d=D
> > e=E
> > f=F
> >
> >   Looking in detail you see that the entries a,b,c are present in both
> > providers, whereas d,e,f are only present in provider 1, and g,h,i only
> in
> > provider 2.
> >   Example Combining PropertyProviders
> >
> > PropertyProvider provider1 = ...
> > PropertyProvider provider2 = ...
> > // aggregate, hereby values from provider 2 override values from
> provider 1
> > PropertyProvider unionOverriding =
> > PropertyProviders.aggregate(AggregationPolicy.OVERRIDE(), provider1,
> > provider2);*System*.out.println("unionOverriding: " +
> > unionOverriding);
> > // ignore duplicates, values present in provider 1 are not overriden
> > by provider 2
> > PropertyProvider unionIgnoringDuplicates =
> > PropertyProviders.aggregate(AggregationPolicy.IGNORE_DUPLICATES(),
> > provider1, provider2);*System*.out.println("unionIgnoringDuplicates: "
> > + unionIgnoringDuplicates);
> > // this variant combines/maps duplicate values into a new value
> > PropertyProvider unionCombined =
> > PropertyProviders.aggregate(AggregationPolicy.COMBINE(), provider1,
> > provider2);*System*.out.println("unionCombined: " + unionCombined);
> > // This variant throws an exception since there are key/value paris in
> > both providers, but with different values*try*{
> >     PropertyProviders.aggregate(AggregationPolicy.EXCEPTION(),
> > provider1, provider2);
> > }*catch*(ConfigException e){
> >     // expected!
> > }
> >
> >   The example above produces the following outpout:
> >   Example Combining PropertyProviders
> >
> > AggregatedPropertyProvider{
> >   (name = dynamicAggregationTests)
> >   a = "[a][A]"
> >   b = "[b][B]"
> >   c = "[c][C]"
> >   d = "[D]"
> >   e = "[E]"
> >   f = "[F]"
> >   g = "[g]"
> >   h = "[h]"
> >   i = "[i]"
> > }
> > unionOverriding: AggregatedPropertyProvider{
> >   (name = <noname>)
> >   a = "A"
> >   b = "B"
> >   c = "C"
> >   d = "D"
> >   e = "E"
> >   f = "F"
> >   g = "g"
> >   h = "h"
> >   i = "i"
> > }
> > unionIgnoringDuplicates: AggregatedPropertyProvider{
> >   (name = <noname>)
> >   a = "a"
> >   b = "b"
> >   c = "c"
> >   d = "D"
> >   e = "E"
> >   f = "F"
> >   g = "g"
> >   h = "h"
> >   i = "i"
> > }
> > unionCombined: AggregatedPropertyProvider{
> >   (name = <noname>)
> >   a = "a,A"
> >   b = "b,B"
> >   c = "c,C"
> >   d = "D"
> >   e = "E"
> >   f = "F"
> >   g = "g"
> >   h = "h"
> >   i = "i"
> > }
> >
> >   No AggregationPolicy is also an interface that can be implemented:
> >   AggregationPolicy Interface
> >
> > @FunctionalInterface*public* *interface* *AggregationPolicy* {
> >     *String* aggregate(*String* key, *String* value1, *String* value2);
> > }
> >
> >   So we can also define our own aggregation strategy using a Lambda
> > expression:
> >   Use a Custom AggregationPolicy
> >
> > PropertyProvider provider1 = ...;
> > PropertyProvider provider2 = ...;
> > PropertyProvider props = PropertyProviders.aggregate(
> >       (k, v1, v2) -> (v1 != null ? v1 : "") + '[' + v2 +
> > "]",
> >       MetaInfo.of("dynamicAggregationTests"),
> >       props1, props2);*System*.out.println(props);
> >
> >   Additionally we also pass here an instance of MetaInfo. The output of
> > this code snippet is as follows:
> >   Listing of dynamic aggregation policy
> >
> > AggregatedPropertyProvider{
> >   (name = dynamicAggregationTests)
> >   a = "[a][A]"
> >   b = "[b][B]"
> >   c = "[c][C]"
> >   d = "[D]"
> >   e = "[E]"
> >   f = "[F]"
> >   g = "[g]"
> >   h = "[h]"
> >   i = "[i]"
> > }
> >
> >   Summarizing the PropertyProviders singleton allows to combine providers
> > in various forms:
> >   Methods provided on PropertyProviders
> >
> > public final class PropertyProviders {
> >
> >     private PropertyProviders() {}
> >
> >     public static PropertyProvider fromArgs(String... args) {
> >     public static PropertyProvider fromArgs(MetaInfo metaInfo, String...
> args)
> > {
> >     public static PropertyProvider fromPaths(AggregationPolicy
> > aggregationPolicy, String... paths) {
> >     public static PropertyProvider fromPaths(String... paths) {
> >     public static PropertyProvider fromPaths(List<String> paths) {
> >     public static PropertyProvider fromPaths(AggregationPolicy
> > aggregationPolicy, List<String> paths) {
> >     public static PropertyProvider fromPaths(MetaInfo metaInfo,
> > List<String> paths) {
> >     public static PropertyProvider fromPaths(AggregationPolicy
> > aggregationPolicy, MetaInfo metaInfo, List<String> paths) {
> >     public static PropertyProvider fromUris(URI... uris) {
> >     public static PropertyProvider fromUris(AggregationPolicy
> > aggregationPolicy, URI... uris) {
> >     public static PropertyProvider fromUris(List<URI> uris) {
> >     public static PropertyProvider fromUris(AggregationPolicy
> > aggregationPolicy, List<URI> uris) {
> >     public static PropertyProvider fromUris(MetaInfo metaInfo, URI...
> uris)
> > {
> >     public static PropertyProvider fromUris(AggregationPolicy
> > aggregationPolicy, MetaInfo metaInfo, URI... uris) {
> >     public static PropertyProvider fromUris(MetaInfo metaInfo, List<URI>
> > uris) {
> >     public static PropertyProvider fromUris(AggregationPolicy
> > aggregationPolicy, MetaInfo metaInfo, List<URI> uris) {
> >     public static PropertyProvider fromMap(Map<String, String> map) {
> >     public static PropertyProvider fromMap(MetaInfo metaInfo,
> > Map<String, String> map) {
> >     public static PropertyProvider empty() {
> >     public static PropertyProvider emptyMutable() {
> >     public static PropertyProvider empty(MetaInfo metaInfo) {
> >     public static PropertyProvider emptyMutable(MetaInfo metaInfo) {
> >     public static PropertyProvider fromEnvironmentProperties() {
> >     public static PropertyProvider fromSystemProperties() {
> >     public static PropertyProvider freezed(PropertyProvider provider) {
> >     public static PropertyProvider aggregate(AggregationPolicy
> > mapping, MetaInfo metaInfo, PropertyProvider... providers){
> >     public static PropertyProvider aggregate(PropertyProvider...
> providers)
> > {
> >     public static PropertyProvider aggregate(List<PropertyProvider>
> > providers) {
> >     public static PropertyProvider aggregate(AggregationPolicy
> > mapping, PropertyProvider... propertyMaps) {
> >     public static PropertyProvider aggregate(AggregationPolicy
> > mapping, List<PropertyProvider> providers) {
> >     public static PropertyProvider mutable(PropertyProvider provider) {
> >     public static PropertyProvider intersected(AggregationPolicy
> > aggregationPolicy, PropertyProvider... providers) {
> >     public static PropertyProvider intersected(PropertyProvider...
> providers)
> > {
> >     public static PropertyProvider subtracted(PropertyProvider target,
> > PropertyProvider... providers) {
> >     public static PropertyProvider filtered(Predicate<String> filter,
> > PropertyProvider provider) {
> >     public static PropertyProvider
> > contextual(Supplier<PropertyProvider> mapSupplier,
> >                                               Supplier<String>
> > isolationKeySupplier) {
> >     public static PropertyProvider delegating(PropertyProvider
> > mainMap, Map<String, String> parentMap) {
> >     public static PropertyProvider replacing(PropertyProvider mainMap,
> > Map<String, String> replacementMap) {
> > }
> >
>

Re: UC: Combining PropertyProvider Instances

[Mark Struberg <st...@yahoo.de>]

I think I need a bit time to understand this. I've tried to grasp it for 2 days now and this might mean that it is too complicated if not even I get it. Why are we doing this so complicated? Pulling in all the database logic into a tool which main target is that it should be really easy to use?
What benefit does this add over hiding those details? e.g. in DeltaSpike I would just add a DatabaseConfigSource and be done. The user would not even see that it magically picks up the values from the db then. 


LieGrue,
strub





> On Tuesday, 2 December 2014, 14:02, Anatole Tresch <an...@apache.org> wrote:
> >    Hi all
> find following an additional part of use cases for comment. Note that this
> area IMO definitively may be improved in several ways. Always keep in mind
> that I try to setup the design guide similarly in parallel. It can be
> easily accessed from the GH mirror:
> 
> https://github.com/apache/incubator-tamaya/blob/master/docs/design/2_CoreConcepts.adoc
> 
> Cheers,
> Anatole
> 
> Combining Property Providers
> 
> Looking at the structures of configuration system used by large companies
> we typically encounter some kind of configuration hierarchies that are
> combined in arbitrary ways. Users of the systems are typically not aware of
> the complexities in this area, since they simply know the possible
> locations, formats and the overriding policies. Framework providers on the
> other side must face the complexities and it would be very useful if Tamaya
> can support here by providing prebuilt functionality that helps
> implementing these aspects. All this leads to the feature set of combining
> property providers. Hereby the following strategies are useful:
> 
>    -
> 
>    aggregating providers, hereby later providers added
>     -
> 
>       override any existing entries from earlier providers
>        -
> 
>       combine conflicting entries from earlier providers, e.g. into a
>       comma-separated structure.
>        -
> 
>       may throw a ConfigExcepotion ig entries are conflicting
>        -
> 
>       may only add entries not yet defined by former providers, preventing
>       entries that are already present to be overwritte
>        -
> 
>       any custom aggregation strategy, which may be a mix of above
>         -
> 
>    intersecting providers
>     -
> 
>    subtracting providers
>     -
> 
>    filtering providers
> 
>   These common functionality is provided by the PropertyProviders
> singleton. Additionally to the base strategies above a MetaInfo instance
> can be passed optionally as well to define the meta information for the
> newly created provider instances. Let’s assume we have two property
> providers with the following data:
>   Provider 1
> 
> a=a
> b=b
> c=c
> g=g
> h=h
> i=i
> 
>   Provider 2
> 
> a=A
> b=B
> c=C
> d=D
> e=E
> f=F
> 
>   Looking in detail you see that the entries a,b,c are present in both
> providers, whereas d,e,f are only present in provider 1, and g,h,i only in
> provider 2.
>   Example Combining PropertyProviders
> 
> PropertyProvider provider1 = ...
> PropertyProvider provider2 = ...
> // aggregate, hereby values from provider 2 override values from provider 1
> PropertyProvider unionOverriding =
> PropertyProviders.aggregate(AggregationPolicy.OVERRIDE(), provider1,
> provider2);*System*.out.println("unionOverriding: " +
> unionOverriding);
> // ignore duplicates, values present in provider 1 are not overriden
> by provider 2
> PropertyProvider unionIgnoringDuplicates =
> PropertyProviders.aggregate(AggregationPolicy.IGNORE_DUPLICATES(),
> provider1, provider2);*System*.out.println("unionIgnoringDuplicates: "
> + unionIgnoringDuplicates);
> // this variant combines/maps duplicate values into a new value
> PropertyProvider unionCombined =
> PropertyProviders.aggregate(AggregationPolicy.COMBINE(), provider1,
> provider2);*System*.out.println("unionCombined: " + unionCombined);
> // This variant throws an exception since there are key/value paris in
> both providers, but with different values*try*{
>     PropertyProviders.aggregate(AggregationPolicy.EXCEPTION(),
> provider1, provider2);
> }*catch*(ConfigException e){
>     // expected!
> }
> 
>   The example above produces the following outpout:
>   Example Combining PropertyProviders
> 
> AggregatedPropertyProvider{
>   (name = dynamicAggregationTests)
>   a = "[a][A]"
>   b = "[b][B]"
>   c = "[c][C]"
>   d = "[D]"
>   e = "[E]"
>   f = "[F]"
>   g = "[g]"
>   h = "[h]"
>   i = "[i]"
> }
> unionOverriding: AggregatedPropertyProvider{
>   (name = <noname>)
>   a = "A"
>   b = "B"
>   c = "C"
>   d = "D"
>   e = "E"
>   f = "F"
>   g = "g"
>   h = "h"
>   i = "i"
> }
> unionIgnoringDuplicates: AggregatedPropertyProvider{
>   (name = <noname>)
>   a = "a"
>   b = "b"
>   c = "c"
>   d = "D"
>   e = "E"
>   f = "F"
>   g = "g"
>   h = "h"
>   i = "i"
> }
> unionCombined: AggregatedPropertyProvider{
>   (name = <noname>)
>   a = "a,A"
>   b = "b,B"
>   c = "c,C"
>   d = "D"
>   e = "E"
>   f = "F"
>   g = "g"
>   h = "h"
>   i = "i"
> }
> 
>   No AggregationPolicy is also an interface that can be implemented:
>   AggregationPolicy Interface
> 
> @FunctionalInterface*public* *interface* *AggregationPolicy* {
>     *String* aggregate(*String* key, *String* value1, *String* value2);
> }
> 
>   So we can also define our own aggregation strategy using a Lambda
> expression:
>   Use a Custom AggregationPolicy
> 
> PropertyProvider provider1 = ...;
> PropertyProvider provider2 = ...;
> PropertyProvider props = PropertyProviders.aggregate(
>       (k, v1, v2) -> (v1 != null ? v1 : "") + '[' + v2 + 
> "]",
>       MetaInfo.of("dynamicAggregationTests"),
>       props1, props2);*System*.out.println(props);
> 
>   Additionally we also pass here an instance of MetaInfo. The output of
> this code snippet is as follows:
>   Listing of dynamic aggregation policy
> 
> AggregatedPropertyProvider{
>   (name = dynamicAggregationTests)
>   a = "[a][A]"
>   b = "[b][B]"
>   c = "[c][C]"
>   d = "[D]"
>   e = "[E]"
>   f = "[F]"
>   g = "[g]"
>   h = "[h]"
>   i = "[i]"
> }
> 
>   Summarizing the PropertyProviders singleton allows to combine providers
> in various forms:
>   Methods provided on PropertyProviders
> 
> public final class PropertyProviders {
> 
>     private PropertyProviders() {}
> 
>     public static PropertyProvider fromArgs(String... args) {
>     public static PropertyProvider fromArgs(MetaInfo metaInfo, String... args) 
> {
>     public static PropertyProvider fromPaths(AggregationPolicy
> aggregationPolicy, String... paths) {
>     public static PropertyProvider fromPaths(String... paths) {
>     public static PropertyProvider fromPaths(List<String> paths) {
>     public static PropertyProvider fromPaths(AggregationPolicy
> aggregationPolicy, List<String> paths) {
>     public static PropertyProvider fromPaths(MetaInfo metaInfo,
> List<String> paths) {
>     public static PropertyProvider fromPaths(AggregationPolicy
> aggregationPolicy, MetaInfo metaInfo, List<String> paths) {
>     public static PropertyProvider fromUris(URI... uris) {
>     public static PropertyProvider fromUris(AggregationPolicy
> aggregationPolicy, URI... uris) {
>     public static PropertyProvider fromUris(List<URI> uris) {
>     public static PropertyProvider fromUris(AggregationPolicy
> aggregationPolicy, List<URI> uris) {
>     public static PropertyProvider fromUris(MetaInfo metaInfo, URI... uris) 
> {
>     public static PropertyProvider fromUris(AggregationPolicy
> aggregationPolicy, MetaInfo metaInfo, URI... uris) {
>     public static PropertyProvider fromUris(MetaInfo metaInfo, List<URI> 
> uris) {
>     public static PropertyProvider fromUris(AggregationPolicy
> aggregationPolicy, MetaInfo metaInfo, List<URI> uris) {
>     public static PropertyProvider fromMap(Map<String, String> map) {
>     public static PropertyProvider fromMap(MetaInfo metaInfo,
> Map<String, String> map) {
>     public static PropertyProvider empty() {
>     public static PropertyProvider emptyMutable() {
>     public static PropertyProvider empty(MetaInfo metaInfo) {
>     public static PropertyProvider emptyMutable(MetaInfo metaInfo) {
>     public static PropertyProvider fromEnvironmentProperties() {
>     public static PropertyProvider fromSystemProperties() {
>     public static PropertyProvider freezed(PropertyProvider provider) {
>     public static PropertyProvider aggregate(AggregationPolicy
> mapping, MetaInfo metaInfo, PropertyProvider... providers){
>     public static PropertyProvider aggregate(PropertyProvider... providers) 
> {
>     public static PropertyProvider aggregate(List<PropertyProvider> 
> providers) {
>     public static PropertyProvider aggregate(AggregationPolicy
> mapping, PropertyProvider... propertyMaps) {
>     public static PropertyProvider aggregate(AggregationPolicy
> mapping, List<PropertyProvider> providers) {
>     public static PropertyProvider mutable(PropertyProvider provider) {
>     public static PropertyProvider intersected(AggregationPolicy
> aggregationPolicy, PropertyProvider... providers) {
>     public static PropertyProvider intersected(PropertyProvider... providers) 
> {
>     public static PropertyProvider subtracted(PropertyProvider target,
> PropertyProvider... providers) {
>     public static PropertyProvider filtered(Predicate<String> filter,
> PropertyProvider provider) {
>     public static PropertyProvider
> contextual(Supplier<PropertyProvider> mapSupplier,
>                                               Supplier<String>
> isolationKeySupplier) {
>     public static PropertyProvider delegating(PropertyProvider
> mainMap, Map<String, String> parentMap) {
>     public static PropertyProvider replacing(PropertyProvider mainMap,
> Map<String, String> replacementMap) {
> }
>

Re: UC: Combining PropertyProvider Instances

["Oliver B. Fischer" <o....@swe-blog.net>]

We have to support simple as well as complex scenarios with Tamaya. For 
us it would be helpfull to define such use cases as

As a administrator of a large DC with X applications and.... I would 
like to do...

Otherwise at least for me it is difficult to understand and to rate a 
given solution.

So I would like to propose such use cases at first before discussing 
everything else. Currently IMHO we focus to much on the already existing 
solution approach.

wdyt?

Oliver

Am 03.12.14 00:54, schrieb Anatole Tresch:
> ​So get into some details of real configuration solutions...
>
>
>     1. ​You may have property sets that have different priorities, but are
>     in the same configuration source.
>     2. ​You may have certain entries from a domain/system config, that
>     should never be overridable by users, whereas others are free to be adapted.
>     3. You max have certain entries that have different priorities not
>     related to the configuration source, but related to an additional meta-info
>     (per entry/area), which defines its own priorization mechanism. E.g. you
>     may have global defaults, domain defaults, ear defaults, app defaults,
>     explicit config related to ear, explicit for application, explicit for
>     tier, server, network zone.
>     4. Additionally you want to define certain packages of configuration
>     (may be from different sources, different files and even formats) where you
>     imply certain checks for integrity and correctness, e.g. your domain
>     configuration should throw an exception if you have conflicting entries,
>     though the entries are read from different configuration sources. In that
>     case overriding is not always the best solutions.
>
> ​I had some discussion with guys throughout the world that run very complex
> setups (Credit Suisse being one of them)​. Experience shows that the way
> configuration is organized is highly differing.
> I agree that organizing everything along priorities works in many cases,
> but unfortunately not in all IMO. Also I prefer to have a intuitive model
> that allows users to explicitly model their configuration (meta-model)
> building blocks, also having the possibility for abstraction (separation of
> concerns), which I think sets and set operations are more convenient that
> splitting everything up to sources and priorities. Since we talk about
> properties, viewing them as sets for me looks quite natural. Set operations
> on top are easy understood and also theoretical powerful enough to cover
> everything out there (do you have the same mathematical background your
> priorities based approach ? ). Also commons-config BTW has similar
> mechanisms in place.
>
> Agree?
>
> Cheers,
> Anatole
>
>
>
>
>
>
>
> Gerhard Petracek
> gerhard.petracek@gmail.com> schrieb am Di., 2. Dez. 2014 um 23:27:
>
> hi @ all,
>> @anatole:
>> please also add >real-world< examples to the document.
>> if we don't have/find them, we need to think about dropping concepts again.
>>
>> fyi:
>> configs in deltaspike get merged based on the key.
>>
>> e.g.:
>> config-source a (priority 1)
>> key1=x
>>
>> config-source b (priority 2)
>> key1=y
>> key2=z
>>
>> ConfigResolver.getPropertyValue("key1") leads to: x
>> ConfigResolver.getPropertyValue("key2") leads to: z
>>
>> imo mark introduced a nice and simple (but powerful) concept with that
>> (esp. since you can re-order even predefined config-source ordinals).
>> deltaspike also uses prefixes for "areas" (internally), but it was never
>> needed to introduce an own concept for that.
>>
>> regards,
>> gerhard
>>
>>
>>
>> 2014-12-02 14:35 GMT+01:00 Oliver B. Fischer <o....@swe-blog.net>:
>>
>>> Yes, it will.
>>> Am 02.12.14 14:19, schrieb Werner Keil:
>>>
>>>> Ok, so at least when generated into common output formats it won't
>>>> show?;-)
>>>>
>>>>
>>>>
>>>>
>>>>
>>> --
>>> N Oliver B. Fischer
>>> A Schönhauser Allee 64, 10437 Berlin, Deutschland/Germany
>>> P +49 30 44793251
>>> M +49 178 7903538
>>> E o.b.fischer@swe-blog.net
>>> S oliver.b.fischer
>>> J oliver.b.fischer@jabber.org
>>> X http://xing.to/obf
>>>
>>>

-- 
N Oliver B. Fischer
A Schönhauser Allee 64, 10437 Berlin, Deutschland/Germany
P +49 30 44793251
M +49 178 7903538
E o.b.fischer@swe-blog.net
S oliver.b.fischer
J oliver.b.fischer@jabber.org
X http://xing.to/obf

Re: UC: Combining PropertyProvider Instances

[Anatole Tresch <at...@gmail.com>]

​So get into some details of real configuration solutions...


   1. ​You may have property sets that have different priorities, but are
   in the same configuration source.
   2. ​You may have certain entries from a domain/system config, that
   should never be overridable by users, whereas others are free to be adapted.
   3. You max have certain entries that have different priorities not
   related to the configuration source, but related to an additional meta-info
   (per entry/area), which defines its own priorization mechanism. E.g. you
   may have global defaults, domain defaults, ear defaults, app defaults,
   explicit config related to ear, explicit for application, explicit for
   tier, server, network zone.
   4. Additionally you want to define certain packages of configuration
   (may be from different sources, different files and even formats) where you
   imply certain checks for integrity and correctness, e.g. your domain
   configuration should throw an exception if you have conflicting entries,
   though the entries are read from different configuration sources. In that
   case overriding is not always the best solutions.

​I had some discussion with guys throughout the world that run very complex
setups (Credit Suisse being one of them)​. Experience shows that the way
configuration is organized is highly differing.
I agree that organizing everything along priorities works in many cases,
but unfortunately not in all IMO. Also I prefer to have a intuitive model
that allows users to explicitly model their configuration (meta-model)
building blocks, also having the possibility for abstraction (separation of
concerns), which I think sets and set operations are more convenient that
splitting everything up to sources and priorities. Since we talk about
properties, viewing them as sets for me looks quite natural. Set operations
on top are easy understood and also theoretical powerful enough to cover
everything out there (do you have the same mathematical background your
priorities based approach ? ). Also commons-config BTW has similar
mechanisms in place.

Agree?

Cheers,
Anatole







Gerhard Petracek
gerhard.petracek@gmail.com> schrieb am Di., 2. Dez. 2014 um 23:27:

hi @ all,
>
> @anatole:
> please also add >real-world< examples to the document.
> if we don't have/find them, we need to think about dropping concepts again.
>
> fyi:
> configs in deltaspike get merged based on the key.
>
> e.g.:
> config-source a (priority 1)
> key1=x
>
> config-source b (priority 2)
> key1=y
> key2=z
>
> ConfigResolver.getPropertyValue("key1") leads to: x
> ConfigResolver.getPropertyValue("key2") leads to: z
>
> imo mark introduced a nice and simple (but powerful) concept with that
> (esp. since you can re-order even predefined config-source ordinals).
> deltaspike also uses prefixes for "areas" (internally), but it was never
> needed to introduce an own concept for that.
>
> regards,
> gerhard
>
>
>
> 2014-12-02 14:35 GMT+01:00 Oliver B. Fischer <o....@swe-blog.net>:
>
> > Yes, it will.
> > Am 02.12.14 14:19, schrieb Werner Keil:
> >
> >> Ok, so at least when generated into common output formats it won't
> >> show?;-)
> >>
> >>
> >>
> >>
> >>
> > --
> > N Oliver B. Fischer
> > A Schönhauser Allee 64, 10437 Berlin, Deutschland/Germany
> > P +49 30 44793251
> > M +49 178 7903538
> > E o.b.fischer@swe-blog.net
> > S oliver.b.fischer
> > J oliver.b.fischer@jabber.org
> > X http://xing.to/obf
> >
> >
>

Re: UC: Combining PropertyProvider Instances

[Gerhard Petracek <ge...@gmail.com>]

hi @ all,

@anatole:
please also add >real-world< examples to the document.
if we don't have/find them, we need to think about dropping concepts again.

fyi:
configs in deltaspike get merged based on the key.

e.g.:
config-source a (priority 1)
key1=x

config-source b (priority 2)
key1=y
key2=z

ConfigResolver.getPropertyValue("key1") leads to: x
ConfigResolver.getPropertyValue("key2") leads to: z

imo mark introduced a nice and simple (but powerful) concept with that
(esp. since you can re-order even predefined config-source ordinals).
deltaspike also uses prefixes for "areas" (internally), but it was never
needed to introduce an own concept for that.

regards,
gerhard



2014-12-02 14:35 GMT+01:00 Oliver B. Fischer <o....@swe-blog.net>:

> Yes, it will.
> Am 02.12.14 14:19, schrieb Werner Keil:
>
>> Ok, so at least when generated into common output formats it won't
>> show?;-)
>>
>>
>>
>>
>>
> --
> N Oliver B. Fischer
> A Schönhauser Allee 64, 10437 Berlin, Deutschland/Germany
> P +49 30 44793251
> M +49 178 7903538
> E o.b.fischer@swe-blog.net
> S oliver.b.fischer
> J oliver.b.fischer@jabber.org
> X http://xing.to/obf
>
>

Re: UC: Combining PropertyProvider Instances

["Oliver B. Fischer" <o....@swe-blog.net>]

Yes, it will.
Am 02.12.14 14:19, schrieb Werner Keil:
> Ok, so at least when generated into common output formats it won't show?;-)
>
>
>
>

-- 
N Oliver B. Fischer
A Schönhauser Allee 64, 10437 Berlin, Deutschland/Germany
P +49 30 44793251
M +49 178 7903538
E o.b.fischer@swe-blog.net
S oliver.b.fischer
J oliver.b.fischer@jabber.org
X http://xing.to/obf

Re: UC: Combining PropertyProvider Instances

[Werner Keil <we...@gmail.com>]

Ok, so at least when generated into common output formats it won't show?;-)






On Tue, Dec 2, 2014 at 2:17 PM, Oliver B. Fischer <o....@swe-blog.net>
wrote:

> It is a variable of Asciidoctor, defined in the top level document.
>
> Am 02.12.14 14:14, schrieb Werner Keil:
>
>  Thanks, is the {Name} at the beginning deliberate, something only GitHub
>> shows, or a bug in the code?
>>
>> Werner
>>
>> On Tue, Dec 2, 2014 at 2:01 PM, Anatole Tresch <an...@apache.org>
>> wrote:
>>
>>      Hi all
>>> find following an additional part of use cases for comment. Note that
>>> this
>>> area IMO definitively may be improved in several ways. Always keep in
>>> mind
>>> that I try to setup the design guide similarly in parallel. It can be
>>> easily accessed from the GH mirror:
>>>
>>>
>>> https://github.com/apache/incubator-tamaya/blob/master/
>>> docs/design/2_CoreConcepts.adoc
>>>
>>> Cheers,
>>> Anatole
>>>
>>> Combining Property Providers
>>>
>>> Looking at the structures of configuration system used by large companies
>>> we typically encounter some kind of configuration hierarchies that are
>>> combined in arbitrary ways. Users of the systems are typically not aware
>>> of
>>> the complexities in this area, since they simply know the possible
>>> locations, formats and the overriding policies. Framework providers on
>>> the
>>> other side must face the complexities and it would be very useful if
>>> Tamaya
>>> can support here by providing prebuilt functionality that helps
>>> implementing these aspects. All this leads to the feature set of
>>> combining
>>> property providers. Hereby the following strategies are useful:
>>>
>>>     -
>>>
>>>     aggregating providers, hereby later providers added
>>>      -
>>>
>>>        override any existing entries from earlier providers
>>>         -
>>>
>>>        combine conflicting entries from earlier providers, e.g. into a
>>>        comma-separated structure.
>>>         -
>>>
>>>        may throw a ConfigExcepotion ig entries are conflicting
>>>         -
>>>
>>>        may only add entries not yet defined by former providers,
>>> preventing
>>>        entries that are already present to be overwritte
>>>         -
>>>
>>>        any custom aggregation strategy, which may be a mix of above
>>>          -
>>>
>>>     intersecting providers
>>>      -
>>>
>>>     subtracting providers
>>>      -
>>>
>>>     filtering providers
>>>
>>>    These common functionality is provided by the PropertyProviders
>>> singleton. Additionally to the base strategies above a MetaInfo instance
>>> can be passed optionally as well to define the meta information for the
>>> newly created provider instances. Let’s assume we have two property
>>> providers with the following data:
>>>    Provider 1
>>>
>>> a=a
>>> b=b
>>> c=c
>>> g=g
>>> h=h
>>> i=i
>>>
>>>    Provider 2
>>>
>>> a=A
>>> b=B
>>> c=C
>>> d=D
>>> e=E
>>> f=F
>>>
>>>    Looking in detail you see that the entries a,b,c are present in both
>>> providers, whereas d,e,f are only present in provider 1, and g,h,i only
>>> in
>>> provider 2.
>>>    Example Combining PropertyProviders
>>>
>>> PropertyProvider provider1 = ...
>>> PropertyProvider provider2 = ...
>>> // aggregate, hereby values from provider 2 override values from
>>> provider 1
>>> PropertyProvider unionOverriding =
>>> PropertyProviders.aggregate(AggregationPolicy.OVERRIDE(), provider1,
>>> provider2);*System*.out.println("unionOverriding: " +
>>> unionOverriding);
>>> // ignore duplicates, values present in provider 1 are not overriden
>>> by provider 2
>>> PropertyProvider unionIgnoringDuplicates =
>>> PropertyProviders.aggregate(AggregationPolicy.IGNORE_DUPLICATES(),
>>> provider1, provider2);*System*.out.println("unionIgnoringDuplicates: "
>>> + unionIgnoringDuplicates);
>>> // this variant combines/maps duplicate values into a new value
>>> PropertyProvider unionCombined =
>>> PropertyProviders.aggregate(AggregationPolicy.COMBINE(), provider1,
>>> provider2);*System*.out.println("unionCombined: " + unionCombined);
>>> // This variant throws an exception since there are key/value paris in
>>> both providers, but with different values*try*{
>>>      PropertyProviders.aggregate(AggregationPolicy.EXCEPTION(),
>>> provider1, provider2);
>>> }*catch*(ConfigException e){
>>>      // expected!
>>> }
>>>
>>>    The example above produces the following outpout:
>>>    Example Combining PropertyProviders
>>>
>>> AggregatedPropertyProvider{
>>>    (name = dynamicAggregationTests)
>>>    a = "[a][A]"
>>>    b = "[b][B]"
>>>    c = "[c][C]"
>>>    d = "[D]"
>>>    e = "[E]"
>>>    f = "[F]"
>>>    g = "[g]"
>>>    h = "[h]"
>>>    i = "[i]"
>>> }
>>> unionOverriding: AggregatedPropertyProvider{
>>>    (name = <noname>)
>>>    a = "A"
>>>    b = "B"
>>>    c = "C"
>>>    d = "D"
>>>    e = "E"
>>>    f = "F"
>>>    g = "g"
>>>    h = "h"
>>>    i = "i"
>>> }
>>> unionIgnoringDuplicates: AggregatedPropertyProvider{
>>>    (name = <noname>)
>>>    a = "a"
>>>    b = "b"
>>>    c = "c"
>>>    d = "D"
>>>    e = "E"
>>>    f = "F"
>>>    g = "g"
>>>    h = "h"
>>>    i = "i"
>>> }
>>> unionCombined: AggregatedPropertyProvider{
>>>    (name = <noname>)
>>>    a = "a,A"
>>>    b = "b,B"
>>>    c = "c,C"
>>>    d = "D"
>>>    e = "E"
>>>    f = "F"
>>>    g = "g"
>>>    h = "h"
>>>    i = "i"
>>> }
>>>
>>>    No AggregationPolicy is also an interface that can be implemented:
>>>    AggregationPolicy Interface
>>>
>>> @FunctionalInterface*public* *interface* *AggregationPolicy* {
>>>      *String* aggregate(*String* key, *String* value1, *String* value2);
>>> }
>>>
>>>    So we can also define our own aggregation strategy using a Lambda
>>> expression:
>>>    Use a Custom AggregationPolicy
>>>
>>> PropertyProvider provider1 = ...;
>>> PropertyProvider provider2 = ...;
>>> PropertyProvider props = PropertyProviders.aggregate(
>>>        (k, v1, v2) -> (v1 != null ? v1 : "") + '[' + v2 + "]",
>>>        MetaInfo.of("dynamicAggregationTests"),
>>>        props1, props2);*System*.out.println(props);
>>>
>>>    Additionally we also pass here an instance of MetaInfo. The output of
>>> this code snippet is as follows:
>>>    Listing of dynamic aggregation policy
>>>
>>> AggregatedPropertyProvider{
>>>    (name = dynamicAggregationTests)
>>>    a = "[a][A]"
>>>    b = "[b][B]"
>>>    c = "[c][C]"
>>>    d = "[D]"
>>>    e = "[E]"
>>>    f = "[F]"
>>>    g = "[g]"
>>>    h = "[h]"
>>>    i = "[i]"
>>> }
>>>
>>>    Summarizing the PropertyProviders singleton allows to combine
>>> providers
>>> in various forms:
>>>    Methods provided on PropertyProviders
>>>
>>> public final class PropertyProviders {
>>>
>>>      private PropertyProviders() {}
>>>
>>>      public static PropertyProvider fromArgs(String... args) {
>>>      public static PropertyProvider fromArgs(MetaInfo metaInfo, String...
>>> args) {
>>>      public static PropertyProvider fromPaths(AggregationPolicy
>>> aggregationPolicy, String... paths) {
>>>      public static PropertyProvider fromPaths(String... paths) {
>>>      public static PropertyProvider fromPaths(List<String> paths) {
>>>      public static PropertyProvider fromPaths(AggregationPolicy
>>> aggregationPolicy, List<String> paths) {
>>>      public static PropertyProvider fromPaths(MetaInfo metaInfo,
>>> List<String> paths) {
>>>      public static PropertyProvider fromPaths(AggregationPolicy
>>> aggregationPolicy, MetaInfo metaInfo, List<String> paths) {
>>>      public static PropertyProvider fromUris(URI... uris) {
>>>      public static PropertyProvider fromUris(AggregationPolicy
>>> aggregationPolicy, URI... uris) {
>>>      public static PropertyProvider fromUris(List<URI> uris) {
>>>      public static PropertyProvider fromUris(AggregationPolicy
>>> aggregationPolicy, List<URI> uris) {
>>>      public static PropertyProvider fromUris(MetaInfo metaInfo, URI...
>>> uris) {
>>>      public static PropertyProvider fromUris(AggregationPolicy
>>> aggregationPolicy, MetaInfo metaInfo, URI... uris) {
>>>      public static PropertyProvider fromUris(MetaInfo metaInfo, List<URI>
>>> uris) {
>>>      public static PropertyProvider fromUris(AggregationPolicy
>>> aggregationPolicy, MetaInfo metaInfo, List<URI> uris) {
>>>      public static PropertyProvider fromMap(Map<String, String> map) {
>>>      public static PropertyProvider fromMap(MetaInfo metaInfo,
>>> Map<String, String> map) {
>>>      public static PropertyProvider empty() {
>>>      public static PropertyProvider emptyMutable() {
>>>      public static PropertyProvider empty(MetaInfo metaInfo) {
>>>      public static PropertyProvider emptyMutable(MetaInfo metaInfo) {
>>>      public static PropertyProvider fromEnvironmentProperties() {
>>>      public static PropertyProvider fromSystemProperties() {
>>>      public static PropertyProvider freezed(PropertyProvider provider) {
>>>      public static PropertyProvider aggregate(AggregationPolicy
>>> mapping, MetaInfo metaInfo, PropertyProvider... providers){
>>>      public static PropertyProvider aggregate(PropertyProvider...
>>> providers) {
>>>      public static PropertyProvider aggregate(List<PropertyProvider>
>>> providers) {
>>>      public static PropertyProvider aggregate(AggregationPolicy
>>> mapping, PropertyProvider... propertyMaps) {
>>>      public static PropertyProvider aggregate(AggregationPolicy
>>> mapping, List<PropertyProvider> providers) {
>>>      public static PropertyProvider mutable(PropertyProvider provider) {
>>>      public static PropertyProvider intersected(AggregationPolicy
>>> aggregationPolicy, PropertyProvider... providers) {
>>>      public static PropertyProvider intersected(PropertyProvider...
>>> providers) {
>>>      public static PropertyProvider subtracted(PropertyProvider target,
>>> PropertyProvider... providers) {
>>>      public static PropertyProvider filtered(Predicate<String> filter,
>>> PropertyProvider provider) {
>>>      public static PropertyProvider
>>> contextual(Supplier<PropertyProvider> mapSupplier,
>>>                                                Supplier<String>
>>> isolationKeySupplier) {
>>>      public static PropertyProvider delegating(PropertyProvider
>>> mainMap, Map<String, String> parentMap) {
>>>      public static PropertyProvider replacing(PropertyProvider mainMap,
>>> Map<String, String> replacementMap) {
>>> }
>>>
>>>
> --
> N Oliver B. Fischer
> A Schönhauser Allee 64, 10437 Berlin, Deutschland/Germany
> P +49 30 44793251
> M +49 178 7903538
> E o.b.fischer@swe-blog.net
> S oliver.b.fischer
> J oliver.b.fischer@jabber.org
> X http://xing.to/obf
>
>

Re: UC: Combining PropertyProvider Instances

["Oliver B. Fischer" <o....@swe-blog.net>]

It is a variable of Asciidoctor, defined in the top level document.

Am 02.12.14 14:14, schrieb Werner Keil:
> Thanks, is the {Name} at the beginning deliberate, something only GitHub
> shows, or a bug in the code?
>
> Werner
>
> On Tue, Dec 2, 2014 at 2:01 PM, Anatole Tresch <an...@apache.org> wrote:
>
>>     Hi all
>> find following an additional part of use cases for comment. Note that this
>> area IMO definitively may be improved in several ways. Always keep in mind
>> that I try to setup the design guide similarly in parallel. It can be
>> easily accessed from the GH mirror:
>>
>>
>> https://github.com/apache/incubator-tamaya/blob/master/docs/design/2_CoreConcepts.adoc
>>
>> Cheers,
>> Anatole
>>
>> Combining Property Providers
>>
>> Looking at the structures of configuration system used by large companies
>> we typically encounter some kind of configuration hierarchies that are
>> combined in arbitrary ways. Users of the systems are typically not aware of
>> the complexities in this area, since they simply know the possible
>> locations, formats and the overriding policies. Framework providers on the
>> other side must face the complexities and it would be very useful if Tamaya
>> can support here by providing prebuilt functionality that helps
>> implementing these aspects. All this leads to the feature set of combining
>> property providers. Hereby the following strategies are useful:
>>
>>     -
>>
>>     aggregating providers, hereby later providers added
>>      -
>>
>>        override any existing entries from earlier providers
>>         -
>>
>>        combine conflicting entries from earlier providers, e.g. into a
>>        comma-separated structure.
>>         -
>>
>>        may throw a ConfigExcepotion ig entries are conflicting
>>         -
>>
>>        may only add entries not yet defined by former providers, preventing
>>        entries that are already present to be overwritte
>>         -
>>
>>        any custom aggregation strategy, which may be a mix of above
>>          -
>>
>>     intersecting providers
>>      -
>>
>>     subtracting providers
>>      -
>>
>>     filtering providers
>>
>>    These common functionality is provided by the PropertyProviders
>> singleton. Additionally to the base strategies above a MetaInfo instance
>> can be passed optionally as well to define the meta information for the
>> newly created provider instances. Let’s assume we have two property
>> providers with the following data:
>>    Provider 1
>>
>> a=a
>> b=b
>> c=c
>> g=g
>> h=h
>> i=i
>>
>>    Provider 2
>>
>> a=A
>> b=B
>> c=C
>> d=D
>> e=E
>> f=F
>>
>>    Looking in detail you see that the entries a,b,c are present in both
>> providers, whereas d,e,f are only present in provider 1, and g,h,i only in
>> provider 2.
>>    Example Combining PropertyProviders
>>
>> PropertyProvider provider1 = ...
>> PropertyProvider provider2 = ...
>> // aggregate, hereby values from provider 2 override values from provider 1
>> PropertyProvider unionOverriding =
>> PropertyProviders.aggregate(AggregationPolicy.OVERRIDE(), provider1,
>> provider2);*System*.out.println("unionOverriding: " +
>> unionOverriding);
>> // ignore duplicates, values present in provider 1 are not overriden
>> by provider 2
>> PropertyProvider unionIgnoringDuplicates =
>> PropertyProviders.aggregate(AggregationPolicy.IGNORE_DUPLICATES(),
>> provider1, provider2);*System*.out.println("unionIgnoringDuplicates: "
>> + unionIgnoringDuplicates);
>> // this variant combines/maps duplicate values into a new value
>> PropertyProvider unionCombined =
>> PropertyProviders.aggregate(AggregationPolicy.COMBINE(), provider1,
>> provider2);*System*.out.println("unionCombined: " + unionCombined);
>> // This variant throws an exception since there are key/value paris in
>> both providers, but with different values*try*{
>>      PropertyProviders.aggregate(AggregationPolicy.EXCEPTION(),
>> provider1, provider2);
>> }*catch*(ConfigException e){
>>      // expected!
>> }
>>
>>    The example above produces the following outpout:
>>    Example Combining PropertyProviders
>>
>> AggregatedPropertyProvider{
>>    (name = dynamicAggregationTests)
>>    a = "[a][A]"
>>    b = "[b][B]"
>>    c = "[c][C]"
>>    d = "[D]"
>>    e = "[E]"
>>    f = "[F]"
>>    g = "[g]"
>>    h = "[h]"
>>    i = "[i]"
>> }
>> unionOverriding: AggregatedPropertyProvider{
>>    (name = <noname>)
>>    a = "A"
>>    b = "B"
>>    c = "C"
>>    d = "D"
>>    e = "E"
>>    f = "F"
>>    g = "g"
>>    h = "h"
>>    i = "i"
>> }
>> unionIgnoringDuplicates: AggregatedPropertyProvider{
>>    (name = <noname>)
>>    a = "a"
>>    b = "b"
>>    c = "c"
>>    d = "D"
>>    e = "E"
>>    f = "F"
>>    g = "g"
>>    h = "h"
>>    i = "i"
>> }
>> unionCombined: AggregatedPropertyProvider{
>>    (name = <noname>)
>>    a = "a,A"
>>    b = "b,B"
>>    c = "c,C"
>>    d = "D"
>>    e = "E"
>>    f = "F"
>>    g = "g"
>>    h = "h"
>>    i = "i"
>> }
>>
>>    No AggregationPolicy is also an interface that can be implemented:
>>    AggregationPolicy Interface
>>
>> @FunctionalInterface*public* *interface* *AggregationPolicy* {
>>      *String* aggregate(*String* key, *String* value1, *String* value2);
>> }
>>
>>    So we can also define our own aggregation strategy using a Lambda
>> expression:
>>    Use a Custom AggregationPolicy
>>
>> PropertyProvider provider1 = ...;
>> PropertyProvider provider2 = ...;
>> PropertyProvider props = PropertyProviders.aggregate(
>>        (k, v1, v2) -> (v1 != null ? v1 : "") + '[' + v2 + "]",
>>        MetaInfo.of("dynamicAggregationTests"),
>>        props1, props2);*System*.out.println(props);
>>
>>    Additionally we also pass here an instance of MetaInfo. The output of
>> this code snippet is as follows:
>>    Listing of dynamic aggregation policy
>>
>> AggregatedPropertyProvider{
>>    (name = dynamicAggregationTests)
>>    a = "[a][A]"
>>    b = "[b][B]"
>>    c = "[c][C]"
>>    d = "[D]"
>>    e = "[E]"
>>    f = "[F]"
>>    g = "[g]"
>>    h = "[h]"
>>    i = "[i]"
>> }
>>
>>    Summarizing the PropertyProviders singleton allows to combine providers
>> in various forms:
>>    Methods provided on PropertyProviders
>>
>> public final class PropertyProviders {
>>
>>      private PropertyProviders() {}
>>
>>      public static PropertyProvider fromArgs(String... args) {
>>      public static PropertyProvider fromArgs(MetaInfo metaInfo, String...
>> args) {
>>      public static PropertyProvider fromPaths(AggregationPolicy
>> aggregationPolicy, String... paths) {
>>      public static PropertyProvider fromPaths(String... paths) {
>>      public static PropertyProvider fromPaths(List<String> paths) {
>>      public static PropertyProvider fromPaths(AggregationPolicy
>> aggregationPolicy, List<String> paths) {
>>      public static PropertyProvider fromPaths(MetaInfo metaInfo,
>> List<String> paths) {
>>      public static PropertyProvider fromPaths(AggregationPolicy
>> aggregationPolicy, MetaInfo metaInfo, List<String> paths) {
>>      public static PropertyProvider fromUris(URI... uris) {
>>      public static PropertyProvider fromUris(AggregationPolicy
>> aggregationPolicy, URI... uris) {
>>      public static PropertyProvider fromUris(List<URI> uris) {
>>      public static PropertyProvider fromUris(AggregationPolicy
>> aggregationPolicy, List<URI> uris) {
>>      public static PropertyProvider fromUris(MetaInfo metaInfo, URI...
>> uris) {
>>      public static PropertyProvider fromUris(AggregationPolicy
>> aggregationPolicy, MetaInfo metaInfo, URI... uris) {
>>      public static PropertyProvider fromUris(MetaInfo metaInfo, List<URI>
>> uris) {
>>      public static PropertyProvider fromUris(AggregationPolicy
>> aggregationPolicy, MetaInfo metaInfo, List<URI> uris) {
>>      public static PropertyProvider fromMap(Map<String, String> map) {
>>      public static PropertyProvider fromMap(MetaInfo metaInfo,
>> Map<String, String> map) {
>>      public static PropertyProvider empty() {
>>      public static PropertyProvider emptyMutable() {
>>      public static PropertyProvider empty(MetaInfo metaInfo) {
>>      public static PropertyProvider emptyMutable(MetaInfo metaInfo) {
>>      public static PropertyProvider fromEnvironmentProperties() {
>>      public static PropertyProvider fromSystemProperties() {
>>      public static PropertyProvider freezed(PropertyProvider provider) {
>>      public static PropertyProvider aggregate(AggregationPolicy
>> mapping, MetaInfo metaInfo, PropertyProvider... providers){
>>      public static PropertyProvider aggregate(PropertyProvider...
>> providers) {
>>      public static PropertyProvider aggregate(List<PropertyProvider>
>> providers) {
>>      public static PropertyProvider aggregate(AggregationPolicy
>> mapping, PropertyProvider... propertyMaps) {
>>      public static PropertyProvider aggregate(AggregationPolicy
>> mapping, List<PropertyProvider> providers) {
>>      public static PropertyProvider mutable(PropertyProvider provider) {
>>      public static PropertyProvider intersected(AggregationPolicy
>> aggregationPolicy, PropertyProvider... providers) {
>>      public static PropertyProvider intersected(PropertyProvider...
>> providers) {
>>      public static PropertyProvider subtracted(PropertyProvider target,
>> PropertyProvider... providers) {
>>      public static PropertyProvider filtered(Predicate<String> filter,
>> PropertyProvider provider) {
>>      public static PropertyProvider
>> contextual(Supplier<PropertyProvider> mapSupplier,
>>                                                Supplier<String>
>> isolationKeySupplier) {
>>      public static PropertyProvider delegating(PropertyProvider
>> mainMap, Map<String, String> parentMap) {
>>      public static PropertyProvider replacing(PropertyProvider mainMap,
>> Map<String, String> replacementMap) {
>> }
>>

-- 
N Oliver B. Fischer
A Schönhauser Allee 64, 10437 Berlin, Deutschland/Germany
P +49 30 44793251
M +49 178 7903538
E o.b.fischer@swe-blog.net
S oliver.b.fischer
J oliver.b.fischer@jabber.org
X http://xing.to/obf

Re: UC: Combining PropertyProvider Instances

[Anatole Tresch <at...@gmail.com>]

When running asciidco from the top level {Name} is resolved. GH does not
honor this on its ad hoc rendering ;(

2014-12-02 14:14 GMT+01:00 Werner Keil <we...@gmail.com>:

> Thanks, is the {Name} at the beginning deliberate, something only GitHub
> shows, or a bug in the code?
>
> Werner
>
> On Tue, Dec 2, 2014 at 2:01 PM, Anatole Tresch <an...@apache.org> wrote:
>
> >    Hi all
> > find following an additional part of use cases for comment. Note that
> this
> > area IMO definitively may be improved in several ways. Always keep in
> mind
> > that I try to setup the design guide similarly in parallel. It can be
> > easily accessed from the GH mirror:
> >
> >
> >
> https://github.com/apache/incubator-tamaya/blob/master/docs/design/2_CoreConcepts.adoc
> >
> > Cheers,
> > Anatole
> >
> > Combining Property Providers
> >
> > Looking at the structures of configuration system used by large companies
> > we typically encounter some kind of configuration hierarchies that are
> > combined in arbitrary ways. Users of the systems are typically not aware
> of
> > the complexities in this area, since they simply know the possible
> > locations, formats and the overriding policies. Framework providers on
> the
> > other side must face the complexities and it would be very useful if
> Tamaya
> > can support here by providing prebuilt functionality that helps
> > implementing these aspects. All this leads to the feature set of
> combining
> > property providers. Hereby the following strategies are useful:
> >
> >    -
> >
> >    aggregating providers, hereby later providers added
> >     -
> >
> >       override any existing entries from earlier providers
> >        -
> >
> >       combine conflicting entries from earlier providers, e.g. into a
> >       comma-separated structure.
> >        -
> >
> >       may throw a ConfigExcepotion ig entries are conflicting
> >        -
> >
> >       may only add entries not yet defined by former providers,
> preventing
> >       entries that are already present to be overwritte
> >        -
> >
> >       any custom aggregation strategy, which may be a mix of above
> >         -
> >
> >    intersecting providers
> >     -
> >
> >    subtracting providers
> >     -
> >
> >    filtering providers
> >
> >   These common functionality is provided by the PropertyProviders
> > singleton. Additionally to the base strategies above a MetaInfo instance
> > can be passed optionally as well to define the meta information for the
> > newly created provider instances. Let’s assume we have two property
> > providers with the following data:
> >   Provider 1
> >
> > a=a
> > b=b
> > c=c
> > g=g
> > h=h
> > i=i
> >
> >   Provider 2
> >
> > a=A
> > b=B
> > c=C
> > d=D
> > e=E
> > f=F
> >
> >   Looking in detail you see that the entries a,b,c are present in both
> > providers, whereas d,e,f are only present in provider 1, and g,h,i only
> in
> > provider 2.
> >   Example Combining PropertyProviders
> >
> > PropertyProvider provider1 = ...
> > PropertyProvider provider2 = ...
> > // aggregate, hereby values from provider 2 override values from
> provider 1
> > PropertyProvider unionOverriding =
> > PropertyProviders.aggregate(AggregationPolicy.OVERRIDE(), provider1,
> > provider2);*System*.out.println("unionOverriding: " +
> > unionOverriding);
> > // ignore duplicates, values present in provider 1 are not overriden
> > by provider 2
> > PropertyProvider unionIgnoringDuplicates =
> > PropertyProviders.aggregate(AggregationPolicy.IGNORE_DUPLICATES(),
> > provider1, provider2);*System*.out.println("unionIgnoringDuplicates: "
> > + unionIgnoringDuplicates);
> > // this variant combines/maps duplicate values into a new value
> > PropertyProvider unionCombined =
> > PropertyProviders.aggregate(AggregationPolicy.COMBINE(), provider1,
> > provider2);*System*.out.println("unionCombined: " + unionCombined);
> > // This variant throws an exception since there are key/value paris in
> > both providers, but with different values*try*{
> >     PropertyProviders.aggregate(AggregationPolicy.EXCEPTION(),
> > provider1, provider2);
> > }*catch*(ConfigException e){
> >     // expected!
> > }
> >
> >   The example above produces the following outpout:
> >   Example Combining PropertyProviders
> >
> > AggregatedPropertyProvider{
> >   (name = dynamicAggregationTests)
> >   a = "[a][A]"
> >   b = "[b][B]"
> >   c = "[c][C]"
> >   d = "[D]"
> >   e = "[E]"
> >   f = "[F]"
> >   g = "[g]"
> >   h = "[h]"
> >   i = "[i]"
> > }
> > unionOverriding: AggregatedPropertyProvider{
> >   (name = <noname>)
> >   a = "A"
> >   b = "B"
> >   c = "C"
> >   d = "D"
> >   e = "E"
> >   f = "F"
> >   g = "g"
> >   h = "h"
> >   i = "i"
> > }
> > unionIgnoringDuplicates: AggregatedPropertyProvider{
> >   (name = <noname>)
> >   a = "a"
> >   b = "b"
> >   c = "c"
> >   d = "D"
> >   e = "E"
> >   f = "F"
> >   g = "g"
> >   h = "h"
> >   i = "i"
> > }
> > unionCombined: AggregatedPropertyProvider{
> >   (name = <noname>)
> >   a = "a,A"
> >   b = "b,B"
> >   c = "c,C"
> >   d = "D"
> >   e = "E"
> >   f = "F"
> >   g = "g"
> >   h = "h"
> >   i = "i"
> > }
> >
> >   No AggregationPolicy is also an interface that can be implemented:
> >   AggregationPolicy Interface
> >
> > @FunctionalInterface*public* *interface* *AggregationPolicy* {
> >     *String* aggregate(*String* key, *String* value1, *String* value2);
> > }
> >
> >   So we can also define our own aggregation strategy using a Lambda
> > expression:
> >   Use a Custom AggregationPolicy
> >
> > PropertyProvider provider1 = ...;
> > PropertyProvider provider2 = ...;
> > PropertyProvider props = PropertyProviders.aggregate(
> >       (k, v1, v2) -> (v1 != null ? v1 : "") + '[' + v2 + "]",
> >       MetaInfo.of("dynamicAggregationTests"),
> >       props1, props2);*System*.out.println(props);
> >
> >   Additionally we also pass here an instance of MetaInfo. The output of
> > this code snippet is as follows:
> >   Listing of dynamic aggregation policy
> >
> > AggregatedPropertyProvider{
> >   (name = dynamicAggregationTests)
> >   a = "[a][A]"
> >   b = "[b][B]"
> >   c = "[c][C]"
> >   d = "[D]"
> >   e = "[E]"
> >   f = "[F]"
> >   g = "[g]"
> >   h = "[h]"
> >   i = "[i]"
> > }
> >
> >   Summarizing the PropertyProviders singleton allows to combine providers
> > in various forms:
> >   Methods provided on PropertyProviders
> >
> > public final class PropertyProviders {
> >
> >     private PropertyProviders() {}
> >
> >     public static PropertyProvider fromArgs(String... args) {
> >     public static PropertyProvider fromArgs(MetaInfo metaInfo, String...
> > args) {
> >     public static PropertyProvider fromPaths(AggregationPolicy
> > aggregationPolicy, String... paths) {
> >     public static PropertyProvider fromPaths(String... paths) {
> >     public static PropertyProvider fromPaths(List<String> paths) {
> >     public static PropertyProvider fromPaths(AggregationPolicy
> > aggregationPolicy, List<String> paths) {
> >     public static PropertyProvider fromPaths(MetaInfo metaInfo,
> > List<String> paths) {
> >     public static PropertyProvider fromPaths(AggregationPolicy
> > aggregationPolicy, MetaInfo metaInfo, List<String> paths) {
> >     public static PropertyProvider fromUris(URI... uris) {
> >     public static PropertyProvider fromUris(AggregationPolicy
> > aggregationPolicy, URI... uris) {
> >     public static PropertyProvider fromUris(List<URI> uris) {
> >     public static PropertyProvider fromUris(AggregationPolicy
> > aggregationPolicy, List<URI> uris) {
> >     public static PropertyProvider fromUris(MetaInfo metaInfo, URI...
> > uris) {
> >     public static PropertyProvider fromUris(AggregationPolicy
> > aggregationPolicy, MetaInfo metaInfo, URI... uris) {
> >     public static PropertyProvider fromUris(MetaInfo metaInfo, List<URI>
> > uris) {
> >     public static PropertyProvider fromUris(AggregationPolicy
> > aggregationPolicy, MetaInfo metaInfo, List<URI> uris) {
> >     public static PropertyProvider fromMap(Map<String, String> map) {
> >     public static PropertyProvider fromMap(MetaInfo metaInfo,
> > Map<String, String> map) {
> >     public static PropertyProvider empty() {
> >     public static PropertyProvider emptyMutable() {
> >     public static PropertyProvider empty(MetaInfo metaInfo) {
> >     public static PropertyProvider emptyMutable(MetaInfo metaInfo) {
> >     public static PropertyProvider fromEnvironmentProperties() {
> >     public static PropertyProvider fromSystemProperties() {
> >     public static PropertyProvider freezed(PropertyProvider provider) {
> >     public static PropertyProvider aggregate(AggregationPolicy
> > mapping, MetaInfo metaInfo, PropertyProvider... providers){
> >     public static PropertyProvider aggregate(PropertyProvider...
> > providers) {
> >     public static PropertyProvider aggregate(List<PropertyProvider>
> > providers) {
> >     public static PropertyProvider aggregate(AggregationPolicy
> > mapping, PropertyProvider... propertyMaps) {
> >     public static PropertyProvider aggregate(AggregationPolicy
> > mapping, List<PropertyProvider> providers) {
> >     public static PropertyProvider mutable(PropertyProvider provider) {
> >     public static PropertyProvider intersected(AggregationPolicy
> > aggregationPolicy, PropertyProvider... providers) {
> >     public static PropertyProvider intersected(PropertyProvider...
> > providers) {
> >     public static PropertyProvider subtracted(PropertyProvider target,
> > PropertyProvider... providers) {
> >     public static PropertyProvider filtered(Predicate<String> filter,
> > PropertyProvider provider) {
> >     public static PropertyProvider
> > contextual(Supplier<PropertyProvider> mapSupplier,
> >                                               Supplier<String>
> > isolationKeySupplier) {
> >     public static PropertyProvider delegating(PropertyProvider
> > mainMap, Map<String, String> parentMap) {
> >     public static PropertyProvider replacing(PropertyProvider mainMap,
> > Map<String, String> replacementMap) {
> > }
> >
>



-- 
*Anatole Tresch*
Java Engineer & Architect, JSR Spec Lead
Glärnischweg 10
CH - 8620 Wetzikon

*Switzerland, Europe Zurich, GMT+1*
*Twitter:  @atsticks*
*Blogs: **http://javaremarkables.blogspot.ch/
<http://javaremarkables.blogspot.ch/>*

*Google: atsticksMobile  +41-76 344 62 79*

Re: UC: Combining PropertyProvider Instances

[Werner Keil <we...@gmail.com>]

Thanks, is the {Name} at the beginning deliberate, something only GitHub
shows, or a bug in the code?

Werner

On Tue, Dec 2, 2014 at 2:01 PM, Anatole Tresch <an...@apache.org> wrote:

>    Hi all
> find following an additional part of use cases for comment. Note that this
> area IMO definitively may be improved in several ways. Always keep in mind
> that I try to setup the design guide similarly in parallel. It can be
> easily accessed from the GH mirror:
>
>
> https://github.com/apache/incubator-tamaya/blob/master/docs/design/2_CoreConcepts.adoc
>
> Cheers,
> Anatole
>
> Combining Property Providers
>
> Looking at the structures of configuration system used by large companies
> we typically encounter some kind of configuration hierarchies that are
> combined in arbitrary ways. Users of the systems are typically not aware of
> the complexities in this area, since they simply know the possible
> locations, formats and the overriding policies. Framework providers on the
> other side must face the complexities and it would be very useful if Tamaya
> can support here by providing prebuilt functionality that helps
> implementing these aspects. All this leads to the feature set of combining
> property providers. Hereby the following strategies are useful:
>
>    -
>
>    aggregating providers, hereby later providers added
>     -
>
>       override any existing entries from earlier providers
>        -
>
>       combine conflicting entries from earlier providers, e.g. into a
>       comma-separated structure.
>        -
>
>       may throw a ConfigExcepotion ig entries are conflicting
>        -
>
>       may only add entries not yet defined by former providers, preventing
>       entries that are already present to be overwritte
>        -
>
>       any custom aggregation strategy, which may be a mix of above
>         -
>
>    intersecting providers
>     -
>
>    subtracting providers
>     -
>
>    filtering providers
>
>   These common functionality is provided by the PropertyProviders
> singleton. Additionally to the base strategies above a MetaInfo instance
> can be passed optionally as well to define the meta information for the
> newly created provider instances. Let’s assume we have two property
> providers with the following data:
>   Provider 1
>
> a=a
> b=b
> c=c
> g=g
> h=h
> i=i
>
>   Provider 2
>
> a=A
> b=B
> c=C
> d=D
> e=E
> f=F
>
>   Looking in detail you see that the entries a,b,c are present in both
> providers, whereas d,e,f are only present in provider 1, and g,h,i only in
> provider 2.
>   Example Combining PropertyProviders
>
> PropertyProvider provider1 = ...
> PropertyProvider provider2 = ...
> // aggregate, hereby values from provider 2 override values from provider 1
> PropertyProvider unionOverriding =
> PropertyProviders.aggregate(AggregationPolicy.OVERRIDE(), provider1,
> provider2);*System*.out.println("unionOverriding: " +
> unionOverriding);
> // ignore duplicates, values present in provider 1 are not overriden
> by provider 2
> PropertyProvider unionIgnoringDuplicates =
> PropertyProviders.aggregate(AggregationPolicy.IGNORE_DUPLICATES(),
> provider1, provider2);*System*.out.println("unionIgnoringDuplicates: "
> + unionIgnoringDuplicates);
> // this variant combines/maps duplicate values into a new value
> PropertyProvider unionCombined =
> PropertyProviders.aggregate(AggregationPolicy.COMBINE(), provider1,
> provider2);*System*.out.println("unionCombined: " + unionCombined);
> // This variant throws an exception since there are key/value paris in
> both providers, but with different values*try*{
>     PropertyProviders.aggregate(AggregationPolicy.EXCEPTION(),
> provider1, provider2);
> }*catch*(ConfigException e){
>     // expected!
> }
>
>   The example above produces the following outpout:
>   Example Combining PropertyProviders
>
> AggregatedPropertyProvider{
>   (name = dynamicAggregationTests)
>   a = "[a][A]"
>   b = "[b][B]"
>   c = "[c][C]"
>   d = "[D]"
>   e = "[E]"
>   f = "[F]"
>   g = "[g]"
>   h = "[h]"
>   i = "[i]"
> }
> unionOverriding: AggregatedPropertyProvider{
>   (name = <noname>)
>   a = "A"
>   b = "B"
>   c = "C"
>   d = "D"
>   e = "E"
>   f = "F"
>   g = "g"
>   h = "h"
>   i = "i"
> }
> unionIgnoringDuplicates: AggregatedPropertyProvider{
>   (name = <noname>)
>   a = "a"
>   b = "b"
>   c = "c"
>   d = "D"
>   e = "E"
>   f = "F"
>   g = "g"
>   h = "h"
>   i = "i"
> }
> unionCombined: AggregatedPropertyProvider{
>   (name = <noname>)
>   a = "a,A"
>   b = "b,B"
>   c = "c,C"
>   d = "D"
>   e = "E"
>   f = "F"
>   g = "g"
>   h = "h"
>   i = "i"
> }
>
>   No AggregationPolicy is also an interface that can be implemented:
>   AggregationPolicy Interface
>
> @FunctionalInterface*public* *interface* *AggregationPolicy* {
>     *String* aggregate(*String* key, *String* value1, *String* value2);
> }
>
>   So we can also define our own aggregation strategy using a Lambda
> expression:
>   Use a Custom AggregationPolicy
>
> PropertyProvider provider1 = ...;
> PropertyProvider provider2 = ...;
> PropertyProvider props = PropertyProviders.aggregate(
>       (k, v1, v2) -> (v1 != null ? v1 : "") + '[' + v2 + "]",
>       MetaInfo.of("dynamicAggregationTests"),
>       props1, props2);*System*.out.println(props);
>
>   Additionally we also pass here an instance of MetaInfo. The output of
> this code snippet is as follows:
>   Listing of dynamic aggregation policy
>
> AggregatedPropertyProvider{
>   (name = dynamicAggregationTests)
>   a = "[a][A]"
>   b = "[b][B]"
>   c = "[c][C]"
>   d = "[D]"
>   e = "[E]"
>   f = "[F]"
>   g = "[g]"
>   h = "[h]"
>   i = "[i]"
> }
>
>   Summarizing the PropertyProviders singleton allows to combine providers
> in various forms:
>   Methods provided on PropertyProviders
>
> public final class PropertyProviders {
>
>     private PropertyProviders() {}
>
>     public static PropertyProvider fromArgs(String... args) {
>     public static PropertyProvider fromArgs(MetaInfo metaInfo, String...
> args) {
>     public static PropertyProvider fromPaths(AggregationPolicy
> aggregationPolicy, String... paths) {
>     public static PropertyProvider fromPaths(String... paths) {
>     public static PropertyProvider fromPaths(List<String> paths) {
>     public static PropertyProvider fromPaths(AggregationPolicy
> aggregationPolicy, List<String> paths) {
>     public static PropertyProvider fromPaths(MetaInfo metaInfo,
> List<String> paths) {
>     public static PropertyProvider fromPaths(AggregationPolicy
> aggregationPolicy, MetaInfo metaInfo, List<String> paths) {
>     public static PropertyProvider fromUris(URI... uris) {
>     public static PropertyProvider fromUris(AggregationPolicy
> aggregationPolicy, URI... uris) {
>     public static PropertyProvider fromUris(List<URI> uris) {
>     public static PropertyProvider fromUris(AggregationPolicy
> aggregationPolicy, List<URI> uris) {
>     public static PropertyProvider fromUris(MetaInfo metaInfo, URI...
> uris) {
>     public static PropertyProvider fromUris(AggregationPolicy
> aggregationPolicy, MetaInfo metaInfo, URI... uris) {
>     public static PropertyProvider fromUris(MetaInfo metaInfo, List<URI>
> uris) {
>     public static PropertyProvider fromUris(AggregationPolicy
> aggregationPolicy, MetaInfo metaInfo, List<URI> uris) {
>     public static PropertyProvider fromMap(Map<String, String> map) {
>     public static PropertyProvider fromMap(MetaInfo metaInfo,
> Map<String, String> map) {
>     public static PropertyProvider empty() {
>     public static PropertyProvider emptyMutable() {
>     public static PropertyProvider empty(MetaInfo metaInfo) {
>     public static PropertyProvider emptyMutable(MetaInfo metaInfo) {
>     public static PropertyProvider fromEnvironmentProperties() {
>     public static PropertyProvider fromSystemProperties() {
>     public static PropertyProvider freezed(PropertyProvider provider) {
>     public static PropertyProvider aggregate(AggregationPolicy
> mapping, MetaInfo metaInfo, PropertyProvider... providers){
>     public static PropertyProvider aggregate(PropertyProvider...
> providers) {
>     public static PropertyProvider aggregate(List<PropertyProvider>
> providers) {
>     public static PropertyProvider aggregate(AggregationPolicy
> mapping, PropertyProvider... propertyMaps) {
>     public static PropertyProvider aggregate(AggregationPolicy
> mapping, List<PropertyProvider> providers) {
>     public static PropertyProvider mutable(PropertyProvider provider) {
>     public static PropertyProvider intersected(AggregationPolicy
> aggregationPolicy, PropertyProvider... providers) {
>     public static PropertyProvider intersected(PropertyProvider...
> providers) {
>     public static PropertyProvider subtracted(PropertyProvider target,
> PropertyProvider... providers) {
>     public static PropertyProvider filtered(Predicate<String> filter,
> PropertyProvider provider) {
>     public static PropertyProvider
> contextual(Supplier<PropertyProvider> mapSupplier,
>                                               Supplier<String>
> isolationKeySupplier) {
>     public static PropertyProvider delegating(PropertyProvider
> mainMap, Map<String, String> parentMap) {
>     public static PropertyProvider replacing(PropertyProvider mainMap,
> Map<String, String> replacementMap) {
> }
>