PropertyFiltering

[Anatole Tresch <an...@apache.org>]

Hi all

I would like to make a *SPI proposal* for PropertyFilter (on API side we
have UnaryOperator<Configuration>). Hereby filters have ordinals and
undergo the same ordering mechanism as PropertySources. Filters get
similarly added to chain of evaluation as PropertySources (implementations
must add @Priority -> do we need a programmatic getOrdinal() to have a more
concise usae here?), with one exceptional rule:
If a filter accidentally has the same ordinal as some PropertySources in
the chain, filters are always executed AFTER PropertySources

So here is the interface:.

/**
 * <p>Interface for filtering the current map of properties during the
evaluation of the chain of PropertySources.
 * Filters can be registered using the {@link
org.apache.tamaya.spi.ServiceContext} or added programmatically
 * by calling {@link
org.apache.tamaya.spi.ConfigurationContext#addPropertyFilter(int,
PropertyFilter)}. The ordinal
 * hereby is defined by the corresponding {@code @Priority} annotation.</p>
 * <p>Filters </p>
 */
public interface PropertyFilter<T>{

    /**
     * Method to check, if a filter is available. Filters that are
currently not available are not considered, when
     * evaluating the current chain of {@link
org.apache.tamaya.spi.PropertySource} instances.
     * @return true, if a filter is available.
     */
    public boolean isAvailable();

    /**
     * Maps the current properties map to the filtered version of the map.
The resulting map will be used as the input
     * for all subsequent handling of PropertySources in the PropertySource
chain.
     * @param currentMap the current property map, not null
     * @return the filtered map, never null.
     */
    Map<String,String> map(Map<String,String> currentMap);

}

WDYT?

Anatole


2014-12-29 17:59 GMT+01:00 Anatole Tresch <an...@apache.org>:

> Whereas overrides are simple with this mechanism. I am asking how we want
> to do other aggregation types, or filtering of values.
>
> *On the API side* filtering is always simple, since we have the
> functional extension points (UnaryOperator<Configuration>), where we can
> easily filter the whole configuration, and the adapters for filtering
> single entries on access.
>
> But let's assume the following use case:
>
> *We have a system low level configuration S, which also contains a few
> entries, I do not want to see on application config A. Given our priority
> based assembly this is not possible, since I only can override the values
> from my parent sources, but never remove them.*
>
> Thinking on this the following ideas come to my mind:
>
> 1) *add PropertySource filters to my chain. I could use the same
> priorities. The bad thing is that it is not clear if the filter should be
> active also, when no source with a similar priority is active and if so if
> it should be before or after, so this will not work.*
> *2) I could define a PropertySource to be filtering, so it is able not
> only to override, but also to remove entries. This "filter" can be
> activated or deactivated depending on the current context. Given that (if
> we have set a correct priority value), all works fine.*
> *3) I could define a separate filter layer which is allowed to filter the
> entries, before they are published (after all overrides were done). This
> would probably be transparently be hooked into the "Configuration Layer",
> which takes the current property sources and creates the final map of
> properties. Then on the final Map the filter service is operating before
> any data is handed out to the callers.*
>
> I think (3) is the best solution here. Nevertheless we should think on how
> this can be done the best way and how the SPI here would look like. Also I
> think, we might have several filters active, e.g. one for logging, one for
> admins, one which considers the current user's role (aka views), ...
>
> So ideas are welcome ;)
>
> Best,
> Anatole
>
>

Re: PropertyFiltering

[Romain Manni-Bucau <rm...@gmail.com>]

Hi Anatole,

what is isAvailable designed for?

Why filters and sources would be in the same chain? shouldnt it be
just some kind of post processors?

Note it is almost impossible to apply filters before read()/get() call
(ie it shouldnt be applied in Configuration building) cause you only
see filled properties and miss defaults for instance which can rely on
filters as well.



Romain Manni-Bucau
@rmannibucau
http://www.tomitribe.com
http://rmannibucau.wordpress.com
https://github.com/rmannibucau


2015-01-01 10:44 GMT+01:00 Anatole Tresch <an...@apache.org>:
> Hi all
>
> I would like to make a *SPI proposal* for PropertyFilter (on API side we
> have UnaryOperator<Configuration>). Hereby filters have ordinals and
> undergo the same ordering mechanism as PropertySources. Filters get
> similarly added to chain of evaluation as PropertySources (implementations
> must add @Priority -> do we need a programmatic getOrdinal() to have a more
> concise usae here?), with one exceptional rule:
> If a filter accidentally has the same ordinal as some PropertySources in
> the chain, filters are always executed AFTER PropertySources
>
> So here is the interface:.
>
> /**
>  * <p>Interface for filtering the current map of properties during the
> evaluation of the chain of PropertySources.
>  * Filters can be registered using the {@link
> org.apache.tamaya.spi.ServiceContext} or added programmatically
>  * by calling {@link
> org.apache.tamaya.spi.ConfigurationContext#addPropertyFilter(int,
> PropertyFilter)}. The ordinal
>  * hereby is defined by the corresponding {@code @Priority} annotation.</p>
>  * <p>Filters </p>
>  */
> public interface PropertyFilter<T>{
>
>     /**
>      * Method to check, if a filter is available. Filters that are
> currently not available are not considered, when
>      * evaluating the current chain of {@link
> org.apache.tamaya.spi.PropertySource} instances.
>      * @return true, if a filter is available.
>      */
>     public boolean isAvailable();
>
>     /**
>      * Maps the current properties map to the filtered version of the map.
> The resulting map will be used as the input
>      * for all subsequent handling of PropertySources in the PropertySource
> chain.
>      * @param currentMap the current property map, not null
>      * @return the filtered map, never null.
>      */
>     Map<String,String> map(Map<String,String> currentMap);
>
> }
>
> WDYT?
>
> Anatole
>
>
> 2014-12-29 17:59 GMT+01:00 Anatole Tresch <an...@apache.org>:
>
>> Whereas overrides are simple with this mechanism. I am asking how we want
>> to do other aggregation types, or filtering of values.
>>
>> *On the API side* filtering is always simple, since we have the
>> functional extension points (UnaryOperator<Configuration>), where we can
>> easily filter the whole configuration, and the adapters for filtering
>> single entries on access.
>>
>> But let's assume the following use case:
>>
>> *We have a system low level configuration S, which also contains a few
>> entries, I do not want to see on application config A. Given our priority
>> based assembly this is not possible, since I only can override the values
>> from my parent sources, but never remove them.*
>>
>> Thinking on this the following ideas come to my mind:
>>
>> 1) *add PropertySource filters to my chain. I could use the same
>> priorities. The bad thing is that it is not clear if the filter should be
>> active also, when no source with a similar priority is active and if so if
>> it should be before or after, so this will not work.*
>> *2) I could define a PropertySource to be filtering, so it is able not
>> only to override, but also to remove entries. This "filter" can be
>> activated or deactivated depending on the current context. Given that (if
>> we have set a correct priority value), all works fine.*
>> *3) I could define a separate filter layer which is allowed to filter the
>> entries, before they are published (after all overrides were done). This
>> would probably be transparently be hooked into the "Configuration Layer",
>> which takes the current property sources and creates the final map of
>> properties. Then on the final Map the filter service is operating before
>> any data is handed out to the callers.*
>>
>> I think (3) is the best solution here. Nevertheless we should think on how
>> this can be done the best way and how the SPI here would look like. Also I
>> think, we might have several filters active, e.g. one for logging, one for
>> admins, one which considers the current user's role (aka views), ...
>>
>> So ideas are welcome ;)
>>
>> Best,
>> Anatole
>>
>>

Re: PropertyFiltering

[Mark Struberg <st...@yahoo.de>]

I thought of EL support as well. 

But once you do this you can forget about all your security ;)
If there is yet another EL injection issue (has been in JSP and JSF already in the past - fixed now) then you would allow attackers to get all your credentials etc.

It is really easy to implement, but I'd rather not do it ;)


LieGrue,
strub





> On Friday, 2 January 2015, 11:07, Anatole Tresch <at...@gmail.com> wrote:
> > It will not be part of core. But it has use cases:
> e.g. you define an entry, like your locale IP-adress once under
> 
> node.ip-addr
> 
> and then reference it, instead of copying the value, e.g. by
> ${config:node.ip-addr}. Similar would be refs to environment and system
> properties...
> At the end users define, what they want. I also have heard some of them
> requesting full EL support! Which is doable, but for most cases it will be
> too much. But given such things are extensions each can extend Tamaya with
> the feature one wants, which is one of the strengths of Tamaya, I think.
> 
> 
> 2015-01-02 10:44 GMT+01:00 Romain Manni-Bucau <rm...@gmail.com>:
> 
>>  ok great - BTW not sure any placeholder is needed here
>> 
>> 
>>  Romain Manni-Bucau
>>  @rmannibucau
>>  http://www.tomitribe.com
>>  http://rmannibucau.wordpress.com
>>  https://github.com/rmannibucau
>> 
>> 
>>  2015-01-02 10:41 GMT+01:00 Anatole Tresch <at...@gmail.com>:
>>  > According to you use case: when you have something like defaultValue =
>>  "My
>>  > ${config:myMappedKey} value." then, yes the filters must be 
> applied as
>>  > well. But that is as well during runtime, e.g. within the 
> Configuration
>>  > implementation, which consumes the things provided by the
>>  > ConfigurationContext. So that should work as well ;)
>>  >
>>  > 2015-01-02 10:22 GMT+01:00 Romain Manni-Bucau 
> <rm...@gmail.com>:
>>  >
>>  >> Ok
>>  >>
>>  >> so it stays runtime.
>>  >>
>>  >> Issue I had in mind was - in pseudo code:
>>  >>
>>  >> @Config(key = "my.foo", defaultValue = 
> "filterX:value")
>>  >> private String str;
>>  >>
>>  >>
>>  >> If my.foo is not in Configuration then filter "X" should 
> still apply
>>  >> but this can only be done lazily.
>>  >>
>>  >>
>>  >>
>>  >>
>>  >> Romain Manni-Bucau
>>  >> @rmannibucau
>>  >> http://www.tomitribe.com
>>  >> http://rmannibucau.wordpress.com
>>  >> https://github.com/rmannibucau
>>  >>
>>  >>
>>  >> 2015-01-02 10:06 GMT+01:00 Anatole Tresch 
> <at...@gmail.com>:
>>  >> > Hi Romain
>>  >> >
>>  >> > I try to explain again and then you can explain us what 
> should not
>>  work
>>  >> ;)
>>  >> >
>>  >> > 1) all property sources are called to evaluate a single 
> value, or all
>>  >> > properties. The ConfigurationContext provides the ordered 
> list or
>>  >> property
>>  >> > sources. Ordering is based on ordinals. So this is the same 
> as with
>>  >> > Deltaspike and will work. PropertySourceProvider allow to 
> dynamically
>>  >> > register propertysources by default.
>>  >> > 2) the value evaluated finally may be go through the filter, 
> e.g. for
>>  >> > resolving placeholders (will be an extension module). Or 
> whatever is
>>  >> > appropriate.
>>  >> >
>>  >> > All other kind of filtering will happen on user level. E.g.
>>  categories,
>>  >> > metadata and similar stuff can be added as separate modules 
> on top of
>>  >> that.
>>  >> >
>>  >> > Looking forward to your thaughts ;)
>>  >> >
>>  >> > Anatole
>>  >> >
>>  >> >
>>  >> >
>>  >> > 2015-01-02 9:48 GMT+01:00 Romain Manni-Bucau 
> <rm...@gmail.com>:
>>  >> >
>>  >> >> Hi Anatole,
>>  >> >>
>>  >> >> just cause I'm not sure I got it: "end of the 
> chain" = ?
>>  >> >>
>>  >> >> If that's after property sources as mentionned 
> earlier it sadly
>>  >> >> doesn't work and you still need them at runtime.
>>  >> >>
>>  >> >>
>>  >> >> Romain Manni-Bucau
>>  >> >> @rmannibucau
>>  >> >> http://www.tomitribe.com
>>  >> >> http://rmannibucau.wordpress.com
>>  >> >> https://github.com/rmannibucau
>>  >> >>
>>  >> >>
>>  >> >> 2015-01-02 8:13 GMT+01:00 Anatole Tresch 
> <at...@gmail.com>:
>>  >> >> > Hi Romain
>>  >> >> >
>>  >> >> > Nobody. (Including me) wants a tree solution ;) be 
> it, we
>>  discussed a
>>  >> >> few things yesterday in the hangout. The only things that 
> change,
>>  will
>>  >> be
>>  >> >> that you get filters at the end of the chain.
>>  >> >> >
>>  >> >> > Additionally I will add extension modules for 
> metadata / categories
>>  >> >> later. But this has no effect on the core api....
>>  >> >> >
>>  >> >> > Anatole
>>  >> >> >
>>  >> >> > -
>>  >> >> > Anatole Tresch
>>  >> >> > Glärnischweg 10
>>  >> >> > 8620 Wetzikon
>>  >> >> > Tel +41 (43) 317 05 30
>>  >> >> > -
>>  >> >> > Send from Mobile
>>  >> >> >
>>  >> >> >> Am 01.01.2015 um 20:42 schrieb Romain 
> Manni-Bucau <
>>  >> >> rmannibucau@gmail.com>:
>>  >> >> >>
>>  >> >> >> I wouldnt start on a tree solution and i dont 
> agree. Aggregation
>>  is
>>  >> fine
>>  >> >> >> and the most performant solution, only issue is 
> not scannable
>>  sources
>>  >> >> and
>>  >> >> >> we can say we dont support them IMO.
>>  >> >> >> Le 1 janv. 2015 20:22, "Anatole 
> Tresch" <at...@gmail.com> a
>>  >> écrit :
>>  >> >> >>
>>  >> >> >>> Hi Romain
>>  >> >> >>>
>>  >> >> >>> Not necessarily, but you simply do mot want 
> to see everything
>>  >> >> everywhere.
>>  >> >> >>> You config needs on domain level are 
> different than on app level.
>>  >> You
>>  >> >> >>> simply do not want to see sll the low level 
> grap. Since we do not
>>  >> have
>>  >> >> >>> named configurations anymore, we need true 
> filtering at the
>>  levels
>>  >> spi
>>  >> >> >>> implementors want it to have.
>>  >> >> >>> Furthernore: evaluating a value with the 
> current override
>>  mechanism
>>  >> is
>>  >> >> too
>>  >> >> >>> costly becausr you are NOT using your chain 
> in an efficient way:
>>  >> when
>>  >> >> you
>>  >> >> >>> have 2 sources a and b. And b defines a 
> value, you always have to
>>  >> >> evaluate
>>  >> >> >>> b, though the fact that b overrides a, so is 
> not relevant for
>>  that
>>  >> >> case.
>>  >> >> >>> Similarly evaluating a non scannable value 
> from a is easy. Just b
>>  >> must
>>  >> >> now,
>>  >> >> >>> which values are not cacheable. That coulb 
> be done by some meta
>>  >> data,
>>  >> >> ie
>>  >> >> >>> with another entry compliant with some 
> metadata scheme.
>>  >> >> >>>
>>  >> >> >>> So you get everything: efficiency , 
> filtering, caching,
>>  >> >> contextuality...
>>  >> >> >>> The only thing that this works is to provide 
> access to the
>>  current
>>  >> >> >>> evaluation chain eg some evaluation context:
>>  >> >> >>> Something like:
>>  >> >> >>>
>>  >> >> >>> Interface evslctx{
>>  >> >> >>>
>>  >> >> >>> Map<string,string> 
> getCurrentProperties();
>>  >> >> >>> String getCurrentValue();
>>  >> >> >>> Boolean contextHasChanged();
>>  >> >> >>> }
>>  >> >> >>>
>>  >> >> >>> Hereby context has changed can be used to 
> flag a change in the
>>  >> >> preceedong
>>  >> >> >>> sources , so the current source can react on 
> the change, eg by
>>  >> >> reevaluating
>>  >> >> >>> getCurrentProperties();
>>  >> >> >>>
>>  >> >> >>> Within the ConfigContext we then can access 
> the topmost
>>  >> propertysource
>>  >> >> >>> available (therefore available and empty are 
> not the same!). On
>>  >> first
>>  >> >> >>> sccess it will go down the gull chsin, 
> whereas later this might
>>  >> pnly be
>>  >> >> >>> necessary when needed...
>>  >> >> >>>
>>  >> >> >>> Anatole
>>  >> >> >>>
>>  >> >> >>> Romain Manni-Bucau 
> <rm...@gmail.com> schrieb am Do., 1.
>>  Jan.
>>  >> >> 2015 um
>>  >> >> >>> 19:46:
>>  >> >> >>>
>>  >> >> >>>> Do you mean we loose the aggregation? 
> Sounds like loosing the
>>  >> >> >>>> configuration management/overview then, 
> are you sure it is good?
>>  >> >> >>>>
>>  >> >> >>>> About filtering I would just care about 
> modifying filters and
>>  >> ignore
>>  >> >> >>>> logging ones = tamaya shouldnt log any 
> value (can log its source
>>  >> if we
>>  >> >> >>>> need) IMO. This logging layer is a very 
> particular need and each
>>  >> app
>>  >> >> >>>> know if a value can be logged or not so 
> adding an abstraction
>>  >> which is
>>  >> >> >>>> not needed in practise seems like 
> overcomplicated to me.
>>  >> >> >>>>
>>  >> >> >>>>
>>  >> >> >>>> Romain Manni-Bucau
>>  >> >> >>>> @rmannibucau
>>  >> >> >>>> http://www.tomitribe.com
>>  >> >> >>>> http://rmannibucau.wordpress.com
>>  >> >> >>>> https://github.com/rmannibucau
>>  >> >> >>>>
>>  >> >> >>>>
>>  >> >> >>>> 2015-01-01 14:18 GMT+01:00 Anatole 
> Tresch <at...@gmail.com>:
>>  >> >> >>>>> See inline...
>>  >> >> >>>>>
>>  >> >> >>>>> 2015-01-01 12:57 GMT+01:00 Mark 
> Struberg <st...@yahoo.de>:
>>  >> >> >>>>>
>>  >> >> >>>>>> If lazy or eager is a matter of 
> each specific Filter. There
>>  is no
>>  >> >> >>> common
>>  >> >> >>>>>> answer to that.
>>  >> >> >>>>>>
>>  >> >> >>>>>> The only reason why we have a 
> problem with it is that the the
>>  >> >> proposed
>>  >> >> >>>>>> interface handles the whole map.
>>  >> >> >>>>>
>>  >> >> >>>>> If we would instead just apply the 
> filtering chain on single
>>  >> >> attributes
>>  >> >> >>>>>> (while they get queried) then 
> all those problems would vanish.
>>  >> >> >>>>>>
>>  >> >> >>>>>
>>  >> >> >>>>> Given what you said above. its an 
> implementation aspect
>>  >> >> >>>>> of the filter
>>  >> >> >>>>> , if a filter passes on a key 
> accessed to the chain for
>>  >> reevaluation,
>>  >> >> >>>>> whereas caching others we have 
> exactly that selective filtering
>>  >> >> >>>>> ... so having control where a filter 
> is placed within the
>>  chain of
>>  >> >> >>>>> property sources for me is the 
> crucial aspect. I also think for
>>  >> the
>>  >> >> >>> sake
>>  >> >> >>>> of
>>  >> >> >>>>> simplicity filters should have the 
> same granularity as property
>>  >> >> >>> sources,
>>  >> >> >>>>> e.g.
>>  >> >> >>>>>
>>  >> >> >>>>> envprops -> sysprops -> 
> domain-cfg -> domainFilter -> app-cfg
>>  ->
>>  >> >> >>>>> securityFilter
>>  >> >> >>>>>
>>  >> >> >>>>> Thats the reason why I started to 
> think to model property
>>  source
>>  >> and
>>  >> >> >>>>> filters very similarly, or even 
> model the property source as
>>  >> property
>>  >> >> >>>> chain
>>  >> >> >>>>> member mapping the current input 
> properties to the output
>>  >> properties
>>  >> >> to
>>  >> >> >>>>> pass to the next member (in that 
> case PropertySource may also
>>  >> >> undergo a
>>  >> >> >>>>> renaming).
>>  >> >> >>>>> Alternatively we could model the 
> property chain also the other
>>  way
>>  >> >> >>> round.
>>  >> >> >>>>> Instead of aggregating all up until 
> we end up with a result, we
>>  >> may
>>  >> >> >>>> simply
>>  >> >> >>>>> ask the highest available (therefore 
> you need an availability
>>  >> flag)
>>  >> >> >>>>> PropertySource for the current 
> configuration. This
>>  implementation
>>  >> >> then
>>  >> >> >>>> can
>>  >> >> >>>>> deliberately decide, if it can 
> provide a key, or it has it to
>>  >> >> delegate
>>  >> >> >>> to
>>  >> >> >>>>> its parent PropertySource:
>>  >> >> >>>>>
>>  >> >> >>>>> Map<String,String> 
> getProperties(PropertySource
>>  >> >> parentPropertySource);
>>  >> >> >>>>> Optional<String> get(String 
> key,
>>  >> >> >>>>> PropertySource 
> parentPropertySource);
>>  >> >> >>>>>
>>  >> >> >>>>> This would solve all: 
> implementation, filtering, caching and
>>  still
>>  >> >> the
>>  >> >> >>>>> ConfigurationContext
>>  >> >> >>>>> around has control over the current 
> chain (he passes the parent
>>  >> >> >>> source)..
>>  >> >> >>>>>
>>  >> >> >>>>> There is btw also a difference 
> between filtering to change
>>  values
>>  >> >> (e.g.
>>  >> >> >>>>>> decrypt ciphered passwords) and 
> filtering for the purpose of
>>  >> >> logging.
>>  >> >> >>>>>>
>>  >> >> >>>>> Agreed. I think should try to 
> list/discuss the use cases in
>>  more
>>  >> >> >>> detail,
>>  >> >> >>>>> when and how configuration should be 
> logged. In most cases I
>>  would
>>  >> >> >>> assume
>>  >> >> >>>>> that we can filter by just adding a 
> top level filter, before
>>  >> passing
>>  >> >> >>> the
>>  >> >> >>>>> config to the logger...
>>  >> >> >>>>>
>>  >> >> >>>>>
>>  >> >> >>>>>
>>  >> >> >>>>>>
>>  >> >> >>>>>> LieGrue,
>>  >> >> >>>>>> strub
>>  >> >> >>>>>>
>>  >> >> >>>>>>
>>  >> >> >>>>>>
>>  >> >> >>>>>>
>>  >> >> >>>>>>
>>  >> >> >>>>>>
>>  >> >> >>>>>>> On Thursday, 1 January 2015, 
> 11:25, Romain Manni-Bucau <
>>  >> >> >>>>>> rmannibucau@gmail.com> wrote:
>>  >> >> >>>>>>>> I think we need both but 
> eager initialization should be the
>>  >> >> >>> default
>>  >> >> >>>>>>> (isScannable = true). All 
> other usages (isScannable = false)
>>  >> should
>>  >> >> >>> be
>>  >> >> >>>>>>> lazy and used with caution 
> (main case I have in mind is JNDI
>>  >> and I
>>  >> >> >>>>>>> wouldnt rely on it really 
> today).
>>  >> >> >>>>>>>
>>  >> >> >>>>>>> What I did in my common impl 
> of conf was to build a
>>  Map<String,
>>  >> >> >>>>>>> String> then 
> "sources" were able to trigger a refresh event
>>  >> making
>>  >> >> >>>>>>> the
>>  >> >> >>>>>>> configuration rebuilding the 
> Map then do a diff and switch if
>>  >> >> >>>>>>> different. This implies only 
> eager eval and makes conf
>>  >> management
>>  >> >> >>> very
>>  >> >> >>>>>>> easy but still allowing 
> updates. Only case it doesnt handle
>>  is
>>  >> lazy
>>  >> >> >>>>>>> eval (JNDI) but this was 
> intended cause harder to
>>  >> >> >>>>>>> manage/control/monitor - 
> view would be partial and then
>>  wrong if
>>  >> >> >>>>>>> allowed.
>>  >> >> >>>>>>>
>>  >> >> >>>>>>>
>>  >> >> >>>>>>>
>>  >> >> >>>>>>> Romain Manni-Bucau
>>  >> >> >>>>>>> @rmannibucau
>>  >> >> >>>>>>> http://www.tomitribe.com
>>  >> >> >>>>>>> 
> http://rmannibucau.wordpress.com
>>  >> >> >>>>>>> 
> https://github.com/rmannibucau
>>  >> >> >>>>>>>
>>  >> >> >>>>>>>
>>  >> >> >>>>>>>
>>  >> >> >>>>>>> 2015-01-01 11:18 GMT+01:00 
> Anatole Tresch <
>>  atsticks@gmail.com>:
>>  >> >> >>>>>>>> I would say lazyness for 
> evaluation is a must (due to
>>  possible
>>  >> >> >>>>>> environment
>>  >> >> >>>>>>>> dependencies properties 
> returned can be different!),
>>  caching of
>>  >> >> >>>>>> aggregated
>>  >> >> >>>>>>>> configuration on some 
> level comes on top. Or did I get
>>  >> something
>>  >> >> >>>>>> wrong...?
>>  >> >> >>>>>>>>
>>  >> >> >>>>>>>>
>>  >> >> >>>>>>>> 2015-01-01 11:11 
> GMT+01:00 Romain Manni-Bucau
>>  >> >> >>>>>>> 
> <rm...@gmail.com>:
>>  >> >> >>>>>>>>
>>  >> >> >>>>>>>>> wouldnt it make it 
> harder to implement - ie not user
>>  friendly
>>  >> but
>>  >> >> >>>> impl
>>  >> >> >>>>>>>>> friendly only?
>>  >> >> >>>>>>>>>
>>  >> >> >>>>>>>>> + not sure lazyness 
> is useful we need an aggregated config
>>  >> anyway
>>  >> >> >>>> (=
>>  >> >> >>>>>>> eager)
>>  >> >> >>>>>>>>>
>>  >> >> >>>>>>>>>
>>  >> >> >>>>>>>>> Romain Manni-Bucau
>>  >> >> >>>>>>>>> @rmannibucau
>>  >> >> >>>>>>>>> 
> http://www.tomitribe.com
>>  >> >> >>>>>>>>> 
> http://rmannibucau.wordpress.com
>>  >> >> >>>>>>>>> 
> https://github.com/rmannibucau
>>  >> >> >>>>>>>>>
>>  >> >> >>>>>>>>>
>>  >> >> >>>>>>>>> 2015-01-01 11:08 
> GMT+01:00 Anatole Tresch <
>>  anatole@apache.org
>>  >> >:
>>  >> >> >>>>>>>>>> Hi all
>>  >> >> >>>>>>>>>>
>>  >> >> >>>>>>>>>> another 
> alternative that comes to my mind, would be to
>>  model
>>  >> >> >>> the
>>  >> >> >>>>>>>>> PropertySource
>>  >> >> >>>>>>>>>> more as a 
> Stream<Map.Entry<String,String>> than a
>>  >> >> >>>>>>> provider:
>>  >> >> >>>>>>>>>>
>>  >> >> >>>>>>>>>>   /**
>>  >> >> >>>>>>>>>>     * 
> <p>Evaluate the current properties as Map, based on
>>  >> >> >>>>>>> the current
>>  >> >> >>>>>>>>>> defined 
> properties. The resulting Map may not        *
>>  return
>>  >> >> >>> all
>>  >> >> >>>>>>> items
>>  >> >> >>>>>>>>>> accessible, when 
> the PropertySource is filtering values
>>  for
>>  >> the
>>  >> >> >>>>>>> context,
>>  >> >> >>>>>>>>>> where the
>>  >> >> >>>>>>>>>>     * 
> PropertySource is available.</p>
>>  >> >> >>>>>>>>>>     * 
> <p>PropertySources that are not filtering data,
>>  >> >> >>>>>>> should either add
>>  >> >> >>>>>>>>> or
>>  >> >> >>>>>>>>>> override 
> entries, but not remove
>>  >> >> >>>>>>>>>>     * any 
> entries provided by the input map.
>>  >> >> >>>>>>>>>>     *
>>  >> >> >>>>>>>>>>     * @return 
> the a corresponding map, never null.
>>  >> >> >>>>>>>>>>     * //X TODO 
> or should we just do getPropertyKeys()?
>>  Think
>>  >> >> >>>>>>> about
>>  >> >> >>>>>>>>>> security (key) 
> vs easier merging (full map)?
>>  >> >> >>>>>>>>>>     */
>>  >> >> >>>>>>>>>>    
> Stream<Map.Entry<String,String>> getProperties(
>>  >> >> >>>>>>>>>> 
> Stream<Map.Entry<String,String>>
>>  >> >> >>>>>>> currentEntriesStream);
>>  >> >> >>>>>>>>>>
>>  >> >> >>>>>>>>>> This allows us 
> to export all collecting and filtering
>>  >> >> >>> activities
>>  >> >> >>>>>>> into the
>>  >> >> >>>>>>>>>> Stream API, and 
> also adding entries is simple:
>>  >> >> >>>>>>>>>>
>>  >> >> >>>>>>>>>> Stream stream = 
> Stream.concat(stream1, stream2);
>>  >> >> >>>>>>>>>>
>>  >> >> >>>>>>>>>> Our code gets 
> bare simple and we profit from all the lazy
>>  >> >> >>>>>>> evaluation
>>  >> >> >>>>>>>>>> features....
>>  >> >> >>>>>>>>>>
>>  >> >> >>>>>>>>>> WDYT?
>>  >> >> >>>>>>>>>>
>>  >> >> >>>>>>>>>>
>>  >> >> >>>>>>>>>>
>>  >> >> >>>>>>>>>> 2015-01-01 10:44 
> GMT+01:00 Anatole Tresch
>>  >> >> >>>>>>> <an...@apache.org>:
>>  >> >> >>>>>>>>>>
>>  >> >> >>>>>>>>>>> Hi all
>>  >> >> >>>>>>>>>>>
>>  >> >> >>>>>>>>>>> I would like 
> to make a *SPI proposal* for PropertyFilter
>>  (on
>>  >> >> >>>>>>> API side we
>>  >> >> >>>>>>>>>>> have 
> UnaryOperator<Configuration>). Hereby filters have
>>  >> >> >>>>>>> ordinals and
>>  >> >> >>>>>>>>>>> undergo the 
> same ordering mechanism as PropertySources.
>>  >> >> >>>>>>> Filters get
>>  >> >> >>>>>>>>>>> similarly 
> added to chain of evaluation as PropertySources
>>  >> >> >>>>>>>>> (implementations
>>  >> >> >>>>>>>>>>> must add 
> @Priority -> do we need a programmatic
>>  >> >> >>>>>>> getOrdinal() to have a
>>  >> >> >>>>>>>>> more
>>  >> >> >>>>>>>>>>> concise usae 
> here?), with one exceptional rule:
>>  >> >> >>>>>>>>>>> If a filter 
> accidentally has the same ordinal as some
>>  >> >> >>>>>>> PropertySources in
>>  >> >> >>>>>>>>>>> the chain, 
> filters are always executed AFTER
>>  PropertySources
>>  >> >> >>>>>>>>>>>
>>  >> >> >>>>>>>>>>> So here is 
> the interface:.
>>  >> >> >>>>>>>>>>>
>>  >> >> >>>>>>>>>>> /**
>>  >> >> >>>>>>>>>>> * 
> <p>Interface for filtering the current map of
>>  >> >> >>>>>>> properties during the
>>  >> >> >>>>>>>>>>> evaluation 
> of the chain of PropertySources.
>>  >> >> >>>>>>>>>>> * Filters 
> can be registered using the {@link
>>  >> >> >>>>>>>>>>> 
> org.apache.tamaya.spi.ServiceContext} or added
>>  >> >> >>>>>>> programmatically
>>  >> >> >>>>>>>>>>> * by calling 
> {@link
>>  >> >> >>>>>>>>>>>
>>  >> >> >>>>>>>
>>  >> org.apache.tamaya.spi.ConfigurationContext#addPropertyFilter(int,
>>  >> >> >>>>>>>>>>> 
> PropertyFilter)}. The ordinal
>>  >> >> >>>>>>>>>>> * hereby is 
> defined by the corresponding {@code
>>  >> >> >>>>>>> @Priority}
>>  >> >> >>>>>>>>> 
> annotation.</p>
>>  >> >> >>>>>>>>>>> * 
> <p>Filters </p>
>>  >> >> >>>>>>>>>>> */
>>  >> >> >>>>>>>>>>> public 
> interface PropertyFilter<T>{
>>  >> >> >>>>>>>>>>>
>>  >> >> >>>>>>>>>>>    /**
>>  >> >> >>>>>>>>>>>     * Method 
> to check, if a filter is available. Filters
>>  >> that
>>  >> >> >>>>>>> are
>>  >> >> >>>>>>>>>>> currently 
> not available are not considered, when
>>  >> >> >>>>>>>>>>>     * 
> evaluating the current chain of {@link
>>  >> >> >>>>>>>>>>> 
> org.apache.tamaya.spi.PropertySource} instances.
>>  >> >> >>>>>>>>>>>     * 
> @return true, if a filter is available.
>>  >> >> >>>>>>>>>>>     */
>>  >> >> >>>>>>>>>>>    public 
> boolean isAvailable();
>>  >> >> >>>>>>>>>>>
>>  >> >> >>>>>>>>>>>    /**
>>  >> >> >>>>>>>>>>>     * Maps 
> the current properties map to the filtered
>>  >> version
>>  >> >> >>>>>>> of the
>>  >> >> >>>>>>>>> map.
>>  >> >> >>>>>>>>>>> The 
> resulting map will be used as the input
>>  >> >> >>>>>>>>>>>     * for 
> all subsequent handling of PropertySources in
>>  the
>>  >> >> >>>>>>>>>>> 
> PropertySource chain.
>>  >> >> >>>>>>>>>>>     * @param 
> currentMap the current property map, not
>>  null
>>  >> >> >>>>>>>>>>>     * 
> @return the filtered map, never null.
>>  >> >> >>>>>>>>>>>     */
>>  >> >> >>>>>>>>>>>    
> Map<String,String> map(Map<String,String>
>>  >> >> >>>>>>> currentMap);
>>  >> >> >>>>>>>>>>>
>>  >> >> >>>>>>>>>>> }
>>  >> >> >>>>>>>>>>>
>>  >> >> >>>>>>>>>>> WDYT?
>>  >> >> >>>>>>>>>>>
>>  >> >> >>>>>>>>>>> Anatole
>>  >> >> >>>>>>>>>>>
>>  >> >> >>>>>>>>>>>
>>  >> >> >>>>>>>>>>> 2014-12-29 
> 17:59 GMT+01:00 Anatole Tresch
>>  >> >> >>>>>>> <an...@apache.org>:
>>  >> >> >>>>>>>>>>>
>>  >> >> >>>>>>>>>>>> Whereas 
> overrides are simple with this mechanism. I am
>>  >> >> >>>>>>> asking how we
>>  >> >> >>>>>>>>> want
>>  >> >> >>>>>>>>>>>> to do 
> other aggregation types, or filtering of values.
>>  >> >> >>>>>>>>>>>>
>>  >> >> >>>>>>>>>>>> *On the 
> API side* filtering is always simple, since we
>>  >> >> >>>>>>> have the
>>  >> >> >>>>>>>>>>>> 
> functional extension points
>>  >> >> >>>>>>> 
> (UnaryOperator<Configuration>), where we
>>  >> >> >>>>>>>>> can
>>  >> >> >>>>>>>>>>>> easily 
> filter the whole configuration, and the adapters
>>  >> >> >>>>>>> for filtering
>>  >> >> >>>>>>>>>>>> single 
> entries on access.
>>  >> >> >>>>>>>>>>>>
>>  >> >> >>>>>>>>>>>> But 
> let's assume the following use case:
>>  >> >> >>>>>>>>>>>>
>>  >> >> >>>>>>>>>>>> *We have 
> a system low level configuration S, which also
>>  >> >> >>>>>>> contains a few
>>  >> >> >>>>>>>>>>>> entries, 
> I do not want to see on application config A.
>>  >> >> >>>>>>> Given our
>>  >> >> >>>>>>>>> priority
>>  >> >> >>>>>>>>>>>> based 
> assembly this is not possible, since I only can
>>  >> >> >>>>>>> override the
>>  >> >> >>>>>>>>> values
>>  >> >> >>>>>>>>>>>> from my 
> parent sources, but never remove them.*
>>  >> >> >>>>>>>>>>>>
>>  >> >> >>>>>>>>>>>> Thinking 
> on this the following ideas come to my mind:
>>  >> >> >>>>>>>>>>>>
>>  >> >> >>>>>>>>>>>> 1) *add 
> PropertySource filters to my chain. I could use
>>  >> >> >>>>>>> the same
>>  >> >> >>>>>>>>>>>> 
> priorities. The bad thing is that it is not clear if the
>>  >> >> >>>>>>> filter should
>>  >> >> >>>>>>>>> be
>>  >> >> >>>>>>>>>>>> active 
> also, when no source with a similar priority is
>>  >> >> >>>>>>> active and if
>>  >> >> >>>>>>>>> so if
>>  >> >> >>>>>>>>>>>> it 
> should be before or after, so this will not work.*
>>  >> >> >>>>>>>>>>>> *2) I 
> could define a PropertySource to be filtering, so
>>  it
>>  >> >> >>>>>>> is able not
>>  >> >> >>>>>>>>>>>> only to 
> override, but also to remove entries. This
>>  >> >> >>>>>>> "filter" can be
>>  >> >> >>>>>>>>>>>> 
> activated or deactivated depending on the current
>>  context.
>>  >> >> >>>>>>> Given that
>>  >> >> >>>>>>>>> (if
>>  >> >> >>>>>>>>>>>> we have 
> set a correct priority value), all works fine.*
>>  >> >> >>>>>>>>>>>> *3) I 
> could define a separate filter layer which is
>>  >> >> >>>>>>> allowed to filter
>>  >> >> >>>>>>>>> the
>>  >> >> >>>>>>>>>>>> entries, 
> before they are published (after all overrides
>>  >> >> >>>>>>> were done).
>>  >> >> >>>>>>>>> This
>>  >> >> >>>>>>>>>>>> would 
> probably be transparently be hooked into the
>>  >> >> >>>>>>> "Configuration
>>  >> >> >>>>>>>>> Layer",
>>  >> >> >>>>>>>>>>>> which 
> takes the current property sources and creates the
>>  >> >> >>>>>>> final map of
>>  >> >> >>>>>>>>>>>> 
> properties. Then on the final Map the filter service is
>>  >> >> >>>>>>> operating
>>  >> >> >>>>>>>>> before
>>  >> >> >>>>>>>>>>>> any data 
> is handed out to the callers.*
>>  >> >> >>>>>>>>>>>>
>>  >> >> >>>>>>>>>>>> I think 
> (3) is the best solution here. Nevertheless we
>>  >> >> >>>>>>> should think on
>>  >> >> >>>>>>>>>>>> how this 
> can be done the best way and how the SPI here
>>  >> >> >>>>>>> would look like.
>>  >> >> >>>>>>>>>>>> Also I 
> think, we might have several filters active, e.g.
>>  >> >> >>>>>>> one for
>>  >> >> >>>>>>>>> logging,
>>  >> >> >>>>>>>>>>>> one for 
> admins, one which considers the current user's
>>  >> >> >>>>>>> role (aka
>>  >> >> >>>>>>>>> views), ...
>>  >> >> >>>>>>>>>>>>
>>  >> >> >>>>>>>>>>>> So ideas 
> are welcome ;)
>>  >> >> >>>>>>>>>>>>
>>  >> >> >>>>>>>>>>>> Best,
>>  >> >> >>>>>>>>>>>> Anatole
>>  >> >> >>>>>>>>>>>>
>>  >> >> >>>>>>>>>>>>
>>  >> >> >>>>>>>>>>>
>>  >> >> >>>>>>>>>
>>  >> >> >>>>>>>>
>>  >> >> >>>>>>>>
>>  >> >> >>>>>>>>
>>  >> >> >>>>>>>> --
>>  >> >> >>>>>>>> *Anatole Tresch*
>>  >> >> >>>>>>>> Java Engineer & 
> Architect, JSR Spec Lead
>>  >> >> >>>>>>>> Glärnischweg 10
>>  >> >> >>>>>>>> CH - 8620 Wetzikon
>>  >> >> >>>>>>>>
>>  >> >> >>>>>>>> *Switzerland, Europe 
> Zurich, GMT+1*
>>  >> >> >>>>>>>> *Twitter:  @atsticks*
>>  >> >> >>>>>>>> *Blogs: 
> **http://javaremarkables.blogspot.ch/
>>  >> >> >>>>>>>> 
> <http://javaremarkables.blogspot.ch/>*
>>  >> >> >>>>>>>>
>>  >> >> >>>>>>>> *Google: atsticksMobile  
> +41-76 344 62 79*
>>  >> >> >>>>>>>
>>  >> >> >>>>>>
>>  >> >> >>>>>
>>  >> >> >>>>>
>>  >> >> >>>>>
>>  >> >> >>>>> --
>>  >> >> >>>>> *Anatole Tresch*
>>  >> >> >>>>> Java Engineer & Architect, JSR 
> Spec Lead
>>  >> >> >>>>> Glärnischweg 10
>>  >> >> >>>>> CH - 8620 Wetzikon
>>  >> >> >>>>>
>>  >> >> >>>>> *Switzerland, Europe Zurich, GMT+1*
>>  >> >> >>>>> *Twitter:  @atsticks*
>>  >> >> >>>>> *Blogs: 
> **http://javaremarkables.blogspot.ch/
>>  >> >> >>>>> 
> <http://javaremarkables.blogspot.ch/>*
>>  >> >> >>>>>
>>  >> >> >>>>> *Google: atsticksMobile  +41-76 344 
> 62 79*
>>  >> >> >>>>
>>  >> >> >>>
>>  >> >>
>>  >> >
>>  >> >
>>  >> >
>>  >> > --
>>  >> > *Anatole Tresch*
>>  >> > Java Engineer & Architect, JSR Spec Lead
>>  >> > Glärnischweg 10
>>  >> > CH - 8620 Wetzikon
>>  >> >
>>  >> > *Switzerland, Europe Zurich, GMT+1*
>>  >> > *Twitter:  @atsticks*
>>  >> > *Blogs: **http://javaremarkables.blogspot.ch/
>>  >> > <http://javaremarkables.blogspot.ch/>*
>>  >> >
>>  >> > *Google: atsticksMobile  +41-76 344 62 79 
> <%2B41-76%20344%2062%2079>*
>>  >>
>>  >
>>  >
>>  >
>>  > --
>>  > *Anatole Tresch*
>>  > Java Engineer & Architect, JSR Spec Lead
>>  > Glärnischweg 10
>>  > CH - 8620 Wetzikon
>>  >
>>  > *Switzerland, Europe Zurich, GMT+1*
>>  > *Twitter:  @atsticks*
>>  > *Blogs: **http://javaremarkables.blogspot.ch/
>>  > <http://javaremarkables.blogspot.ch/>*
>>  >
>>  > *Google: atsticksMobile  +41-76 344 62 79*
>> 
> 
> 
> 
> -- 
> *Anatole Tresch*
> Java Engineer & Architect, JSR Spec Lead
> Glärnischweg 10
> CH - 8620 Wetzikon
> 
> *Switzerland, Europe Zurich, GMT+1*
> *Twitter:  @atsticks*
> *Blogs: **http://javaremarkables.blogspot.ch/
> <http://javaremarkables.blogspot.ch/>*
> 
> *Google: atsticksMobile  +41-76 344 62 79*
>

Re: PropertyFiltering

[Romain Manni-Bucau <rm...@gmail.com>]

I didnt say it was a bad feature, just it is not needed.

Let me explain further - I'll use annotations since it is easier for
me but it is the same if using the programmatic solution.

A standard default:

@Config(key = ..., defaultValue = "some nice default")


You can't distinguish it from:

@Config(key = ..., defaultValue = "filter:some nice default")

which could be resolved to "foo" if the filter desires.

${} could just be a shortcut for one particular filter (ie system/envrt props?)

Also note the filter needs to have the key cause the value can depend
on the key (or even default to the key in some cases).


That's why I'd go for

Filter {
   boolean supports(key, value)
   String applyOrAnyNiceName(key, value)
}

Of course the usage would be then while (valueChanged) { doFilterTheValue(); }





Romain Manni-Bucau
@rmannibucau
http://www.tomitribe.com
http://rmannibucau.wordpress.com
https://github.com/rmannibucau


2015-01-02 11:04 GMT+01:00 Anatole Tresch <at...@gmail.com>:
> It will not be part of core. But it has use cases:
> e.g. you define an entry, like your locale IP-adress once under
>
> node.ip-addr
>
> and then reference it, instead of copying the value, e.g. by
> ${config:node.ip-addr}. Similar would be refs to environment and system
> properties...
> At the end users define, what they want. I also have heard some of them
> requesting full EL support! Which is doable, but for most cases it will be
> too much. But given such things are extensions each can extend Tamaya with
> the feature one wants, which is one of the strengths of Tamaya, I think.
>
>
> 2015-01-02 10:44 GMT+01:00 Romain Manni-Bucau <rm...@gmail.com>:
>
>> ok great - BTW not sure any placeholder is needed here
>>
>>
>> Romain Manni-Bucau
>> @rmannibucau
>> http://www.tomitribe.com
>> http://rmannibucau.wordpress.com
>> https://github.com/rmannibucau
>>
>>
>> 2015-01-02 10:41 GMT+01:00 Anatole Tresch <at...@gmail.com>:
>> > According to you use case: when you have something like defaultValue =
>> "My
>> > ${config:myMappedKey} value." then, yes the filters must be applied as
>> > well. But that is as well during runtime, e.g. within the Configuration
>> > implementation, which consumes the things provided by the
>> > ConfigurationContext. So that should work as well ;)
>> >
>> > 2015-01-02 10:22 GMT+01:00 Romain Manni-Bucau <rm...@gmail.com>:
>> >
>> >> Ok
>> >>
>> >> so it stays runtime.
>> >>
>> >> Issue I had in mind was - in pseudo code:
>> >>
>> >> @Config(key = "my.foo", defaultValue = "filterX:value")
>> >> private String str;
>> >>
>> >>
>> >> If my.foo is not in Configuration then filter "X" should still apply
>> >> but this can only be done lazily.
>> >>
>> >>
>> >>
>> >>
>> >> Romain Manni-Bucau
>> >> @rmannibucau
>> >> http://www.tomitribe.com
>> >> http://rmannibucau.wordpress.com
>> >> https://github.com/rmannibucau
>> >>
>> >>
>> >> 2015-01-02 10:06 GMT+01:00 Anatole Tresch <at...@gmail.com>:
>> >> > Hi Romain
>> >> >
>> >> > I try to explain again and then you can explain us what should not
>> work
>> >> ;)
>> >> >
>> >> > 1) all property sources are called to evaluate a single value, or all
>> >> > properties. The ConfigurationContext provides the ordered list or
>> >> property
>> >> > sources. Ordering is based on ordinals. So this is the same as with
>> >> > Deltaspike and will work. PropertySourceProvider allow to dynamically
>> >> > register propertysources by default.
>> >> > 2) the value evaluated finally may be go through the filter, e.g. for
>> >> > resolving placeholders (will be an extension module). Or whatever is
>> >> > appropriate.
>> >> >
>> >> > All other kind of filtering will happen on user level. E.g.
>> categories,
>> >> > metadata and similar stuff can be added as separate modules on top of
>> >> that.
>> >> >
>> >> > Looking forward to your thaughts ;)
>> >> >
>> >> > Anatole
>> >> >
>> >> >
>> >> >
>> >> > 2015-01-02 9:48 GMT+01:00 Romain Manni-Bucau <rm...@gmail.com>:
>> >> >
>> >> >> Hi Anatole,
>> >> >>
>> >> >> just cause I'm not sure I got it: "end of the chain" = ?
>> >> >>
>> >> >> If that's after property sources as mentionned earlier it sadly
>> >> >> doesn't work and you still need them at runtime.
>> >> >>
>> >> >>
>> >> >> Romain Manni-Bucau
>> >> >> @rmannibucau
>> >> >> http://www.tomitribe.com
>> >> >> http://rmannibucau.wordpress.com
>> >> >> https://github.com/rmannibucau
>> >> >>
>> >> >>
>> >> >> 2015-01-02 8:13 GMT+01:00 Anatole Tresch <at...@gmail.com>:
>> >> >> > Hi Romain
>> >> >> >
>> >> >> > Nobody. (Including me) wants a tree solution ;) be it, we
>> discussed a
>> >> >> few things yesterday in the hangout. The only things that change,
>> will
>> >> be
>> >> >> that you get filters at the end of the chain.
>> >> >> >
>> >> >> > Additionally I will add extension modules for metadata / categories
>> >> >> later. But this has no effect on the core api....
>> >> >> >
>> >> >> > Anatole
>> >> >> >
>> >> >> > -
>> >> >> > Anatole Tresch
>> >> >> > Glärnischweg 10
>> >> >> > 8620 Wetzikon
>> >> >> > Tel +41 (43) 317 05 30
>> >> >> > -
>> >> >> > Send from Mobile
>> >> >> >
>> >> >> >> Am 01.01.2015 um 20:42 schrieb Romain Manni-Bucau <
>> >> >> rmannibucau@gmail.com>:
>> >> >> >>
>> >> >> >> I wouldnt start on a tree solution and i dont agree. Aggregation
>> is
>> >> fine
>> >> >> >> and the most performant solution, only issue is not scannable
>> sources
>> >> >> and
>> >> >> >> we can say we dont support them IMO.
>> >> >> >> Le 1 janv. 2015 20:22, "Anatole Tresch" <at...@gmail.com> a
>> >> écrit :
>> >> >> >>
>> >> >> >>> Hi Romain
>> >> >> >>>
>> >> >> >>> Not necessarily, but you simply do mot want to see everything
>> >> >> everywhere.
>> >> >> >>> You config needs on domain level are different than on app level.
>> >> You
>> >> >> >>> simply do not want to see sll the low level grap. Since we do not
>> >> have
>> >> >> >>> named configurations anymore, we need true filtering at the
>> levels
>> >> spi
>> >> >> >>> implementors want it to have.
>> >> >> >>> Furthernore: evaluating a value with the current override
>> mechanism
>> >> is
>> >> >> too
>> >> >> >>> costly becausr you are NOT using your chain in an efficient way:
>> >> when
>> >> >> you
>> >> >> >>> have 2 sources a and b. And b defines a value, you always have to
>> >> >> evaluate
>> >> >> >>> b, though the fact that b overrides a, so is not relevant for
>> that
>> >> >> case.
>> >> >> >>> Similarly evaluating a non scannable value from a is easy. Just b
>> >> must
>> >> >> now,
>> >> >> >>> which values are not cacheable. That coulb be done by some meta
>> >> data,
>> >> >> ie
>> >> >> >>> with another entry compliant with some metadata scheme.
>> >> >> >>>
>> >> >> >>> So you get everything: efficiency , filtering, caching,
>> >> >> contextuality...
>> >> >> >>> The only thing that this works is to provide access to the
>> current
>> >> >> >>> evaluation chain eg some evaluation context:
>> >> >> >>> Something like:
>> >> >> >>>
>> >> >> >>> Interface evslctx{
>> >> >> >>>
>> >> >> >>> Map<string,string> getCurrentProperties();
>> >> >> >>> String getCurrentValue();
>> >> >> >>> Boolean contextHasChanged();
>> >> >> >>> }
>> >> >> >>>
>> >> >> >>> Hereby context has changed can be used to flag a change in the
>> >> >> preceedong
>> >> >> >>> sources , so the current source can react on the change, eg by
>> >> >> reevaluating
>> >> >> >>> getCurrentProperties();
>> >> >> >>>
>> >> >> >>> Within the ConfigContext we then can access the topmost
>> >> propertysource
>> >> >> >>> available (therefore available and empty are not the same!). On
>> >> first
>> >> >> >>> sccess it will go down the gull chsin, whereas later this might
>> >> pnly be
>> >> >> >>> necessary when needed...
>> >> >> >>>
>> >> >> >>> Anatole
>> >> >> >>>
>> >> >> >>> Romain Manni-Bucau <rm...@gmail.com> schrieb am Do., 1.
>> Jan.
>> >> >> 2015 um
>> >> >> >>> 19:46:
>> >> >> >>>
>> >> >> >>>> Do you mean we loose the aggregation? Sounds like loosing the
>> >> >> >>>> configuration management/overview then, are you sure it is good?
>> >> >> >>>>
>> >> >> >>>> About filtering I would just care about modifying filters and
>> >> ignore
>> >> >> >>>> logging ones = tamaya shouldnt log any value (can log its source
>> >> if we
>> >> >> >>>> need) IMO. This logging layer is a very particular need and each
>> >> app
>> >> >> >>>> know if a value can be logged or not so adding an abstraction
>> >> which is
>> >> >> >>>> not needed in practise seems like overcomplicated to me.
>> >> >> >>>>
>> >> >> >>>>
>> >> >> >>>> Romain Manni-Bucau
>> >> >> >>>> @rmannibucau
>> >> >> >>>> http://www.tomitribe.com
>> >> >> >>>> http://rmannibucau.wordpress.com
>> >> >> >>>> https://github.com/rmannibucau
>> >> >> >>>>
>> >> >> >>>>
>> >> >> >>>> 2015-01-01 14:18 GMT+01:00 Anatole Tresch <at...@gmail.com>:
>> >> >> >>>>> See inline...
>> >> >> >>>>>
>> >> >> >>>>> 2015-01-01 12:57 GMT+01:00 Mark Struberg <st...@yahoo.de>:
>> >> >> >>>>>
>> >> >> >>>>>> If lazy or eager is a matter of each specific Filter. There
>> is no
>> >> >> >>> common
>> >> >> >>>>>> answer to that.
>> >> >> >>>>>>
>> >> >> >>>>>> The only reason why we have a problem with it is that the the
>> >> >> proposed
>> >> >> >>>>>> interface handles the whole map.
>> >> >> >>>>>
>> >> >> >>>>> If we would instead just apply the filtering chain on single
>> >> >> attributes
>> >> >> >>>>>> (while they get queried) then all those problems would vanish.
>> >> >> >>>>>>
>> >> >> >>>>>
>> >> >> >>>>> Given what you said above. its an implementation aspect
>> >> >> >>>>> of the filter
>> >> >> >>>>> , if a filter passes on a key accessed to the chain for
>> >> reevaluation,
>> >> >> >>>>> whereas caching others we have exactly that selective filtering
>> >> >> >>>>> ... so having control where a filter is placed within the
>> chain of
>> >> >> >>>>> property sources for me is the crucial aspect. I also think for
>> >> the
>> >> >> >>> sake
>> >> >> >>>> of
>> >> >> >>>>> simplicity filters should have the same granularity as property
>> >> >> >>> sources,
>> >> >> >>>>> e.g.
>> >> >> >>>>>
>> >> >> >>>>> envprops -> sysprops -> domain-cfg -> domainFilter -> app-cfg
>> ->
>> >> >> >>>>> securityFilter
>> >> >> >>>>>
>> >> >> >>>>> Thats the reason why I started to think to model property
>> source
>> >> and
>> >> >> >>>>> filters very similarly, or even model the property source as
>> >> property
>> >> >> >>>> chain
>> >> >> >>>>> member mapping the current input properties to the output
>> >> properties
>> >> >> to
>> >> >> >>>>> pass to the next member (in that case PropertySource may also
>> >> >> undergo a
>> >> >> >>>>> renaming).
>> >> >> >>>>> Alternatively we could model the property chain also the other
>> way
>> >> >> >>> round.
>> >> >> >>>>> Instead of aggregating all up until we end up with a result, we
>> >> may
>> >> >> >>>> simply
>> >> >> >>>>> ask the highest available (therefore you need an availability
>> >> flag)
>> >> >> >>>>> PropertySource for the current configuration. This
>> implementation
>> >> >> then
>> >> >> >>>> can
>> >> >> >>>>> deliberately decide, if it can provide a key, or it has it to
>> >> >> delegate
>> >> >> >>> to
>> >> >> >>>>> its parent PropertySource:
>> >> >> >>>>>
>> >> >> >>>>> Map<String,String> getProperties(PropertySource
>> >> >> parentPropertySource);
>> >> >> >>>>> Optional<String> get(String key,
>> >> >> >>>>> PropertySource parentPropertySource);
>> >> >> >>>>>
>> >> >> >>>>> This would solve all: implementation, filtering, caching and
>> still
>> >> >> the
>> >> >> >>>>> ConfigurationContext
>> >> >> >>>>> around has control over the current chain (he passes the parent
>> >> >> >>> source)..
>> >> >> >>>>>
>> >> >> >>>>> There is btw also a difference between filtering to change
>> values
>> >> >> (e.g.
>> >> >> >>>>>> decrypt ciphered passwords) and filtering for the purpose of
>> >> >> logging.
>> >> >> >>>>>>
>> >> >> >>>>> Agreed. I think should try to list/discuss the use cases in
>> more
>> >> >> >>> detail,
>> >> >> >>>>> when and how configuration should be logged. In most cases I
>> would
>> >> >> >>> assume
>> >> >> >>>>> that we can filter by just adding a top level filter, before
>> >> passing
>> >> >> >>> the
>> >> >> >>>>> config to the logger...
>> >> >> >>>>>
>> >> >> >>>>>
>> >> >> >>>>>
>> >> >> >>>>>>
>> >> >> >>>>>> LieGrue,
>> >> >> >>>>>> strub
>> >> >> >>>>>>
>> >> >> >>>>>>
>> >> >> >>>>>>
>> >> >> >>>>>>
>> >> >> >>>>>>
>> >> >> >>>>>>
>> >> >> >>>>>>> On Thursday, 1 January 2015, 11:25, Romain Manni-Bucau <
>> >> >> >>>>>> rmannibucau@gmail.com> wrote:
>> >> >> >>>>>>>> I think we need both but eager initialization should be the
>> >> >> >>> default
>> >> >> >>>>>>> (isScannable = true). All other usages (isScannable = false)
>> >> should
>> >> >> >>> be
>> >> >> >>>>>>> lazy and used with caution (main case I have in mind is JNDI
>> >> and I
>> >> >> >>>>>>> wouldnt rely on it really today).
>> >> >> >>>>>>>
>> >> >> >>>>>>> What I did in my common impl of conf was to build a
>> Map<String,
>> >> >> >>>>>>> String> then "sources" were able to trigger a refresh event
>> >> making
>> >> >> >>>>>>> the
>> >> >> >>>>>>> configuration rebuilding the Map then do a diff and switch if
>> >> >> >>>>>>> different. This implies only eager eval and makes conf
>> >> management
>> >> >> >>> very
>> >> >> >>>>>>> easy but still allowing updates. Only case it doesnt handle
>> is
>> >> lazy
>> >> >> >>>>>>> eval (JNDI) but this was intended cause harder to
>> >> >> >>>>>>> manage/control/monitor - view would be partial and then
>> wrong if
>> >> >> >>>>>>> allowed.
>> >> >> >>>>>>>
>> >> >> >>>>>>>
>> >> >> >>>>>>>
>> >> >> >>>>>>> Romain Manni-Bucau
>> >> >> >>>>>>> @rmannibucau
>> >> >> >>>>>>> http://www.tomitribe.com
>> >> >> >>>>>>> http://rmannibucau.wordpress.com
>> >> >> >>>>>>> https://github.com/rmannibucau
>> >> >> >>>>>>>
>> >> >> >>>>>>>
>> >> >> >>>>>>>
>> >> >> >>>>>>> 2015-01-01 11:18 GMT+01:00 Anatole Tresch <
>> atsticks@gmail.com>:
>> >> >> >>>>>>>> I would say lazyness for evaluation is a must (due to
>> possible
>> >> >> >>>>>> environment
>> >> >> >>>>>>>> dependencies properties returned can be different!),
>> caching of
>> >> >> >>>>>> aggregated
>> >> >> >>>>>>>> configuration on some level comes on top. Or did I get
>> >> something
>> >> >> >>>>>> wrong...?
>> >> >> >>>>>>>>
>> >> >> >>>>>>>>
>> >> >> >>>>>>>> 2015-01-01 11:11 GMT+01:00 Romain Manni-Bucau
>> >> >> >>>>>>> <rm...@gmail.com>:
>> >> >> >>>>>>>>
>> >> >> >>>>>>>>> wouldnt it make it harder to implement - ie not user
>> friendly
>> >> but
>> >> >> >>>> impl
>> >> >> >>>>>>>>> friendly only?
>> >> >> >>>>>>>>>
>> >> >> >>>>>>>>> + not sure lazyness is useful we need an aggregated config
>> >> anyway
>> >> >> >>>> (=
>> >> >> >>>>>>> eager)
>> >> >> >>>>>>>>>
>> >> >> >>>>>>>>>
>> >> >> >>>>>>>>> Romain Manni-Bucau
>> >> >> >>>>>>>>> @rmannibucau
>> >> >> >>>>>>>>> http://www.tomitribe.com
>> >> >> >>>>>>>>> http://rmannibucau.wordpress.com
>> >> >> >>>>>>>>> https://github.com/rmannibucau
>> >> >> >>>>>>>>>
>> >> >> >>>>>>>>>
>> >> >> >>>>>>>>> 2015-01-01 11:08 GMT+01:00 Anatole Tresch <
>> anatole@apache.org
>> >> >:
>> >> >> >>>>>>>>>> Hi all
>> >> >> >>>>>>>>>>
>> >> >> >>>>>>>>>> another alternative that comes to my mind, would be to
>> model
>> >> >> >>> the
>> >> >> >>>>>>>>> PropertySource
>> >> >> >>>>>>>>>> more as a Stream<Map.Entry<String,String>> than a
>> >> >> >>>>>>> provider:
>> >> >> >>>>>>>>>>
>> >> >> >>>>>>>>>>   /**
>> >> >> >>>>>>>>>>     * <p>Evaluate the current properties as Map, based on
>> >> >> >>>>>>> the current
>> >> >> >>>>>>>>>> defined properties. The resulting Map may not        *
>> return
>> >> >> >>> all
>> >> >> >>>>>>> items
>> >> >> >>>>>>>>>> accessible, when the PropertySource is filtering values
>> for
>> >> the
>> >> >> >>>>>>> context,
>> >> >> >>>>>>>>>> where the
>> >> >> >>>>>>>>>>     * PropertySource is available.</p>
>> >> >> >>>>>>>>>>     * <p>PropertySources that are not filtering data,
>> >> >> >>>>>>> should either add
>> >> >> >>>>>>>>> or
>> >> >> >>>>>>>>>> override entries, but not remove
>> >> >> >>>>>>>>>>     * any entries provided by the input map.
>> >> >> >>>>>>>>>>     *
>> >> >> >>>>>>>>>>     * @return the a corresponding map, never null.
>> >> >> >>>>>>>>>>     * //X TODO or should we just do getPropertyKeys()?
>> Think
>> >> >> >>>>>>> about
>> >> >> >>>>>>>>>> security (key) vs easier merging (full map)?
>> >> >> >>>>>>>>>>     */
>> >> >> >>>>>>>>>>    Stream<Map.Entry<String,String>> getProperties(
>> >> >> >>>>>>>>>> Stream<Map.Entry<String,String>>
>> >> >> >>>>>>> currentEntriesStream);
>> >> >> >>>>>>>>>>
>> >> >> >>>>>>>>>> This allows us to export all collecting and filtering
>> >> >> >>> activities
>> >> >> >>>>>>> into the
>> >> >> >>>>>>>>>> Stream API, and also adding entries is simple:
>> >> >> >>>>>>>>>>
>> >> >> >>>>>>>>>> Stream stream = Stream.concat(stream1, stream2);
>> >> >> >>>>>>>>>>
>> >> >> >>>>>>>>>> Our code gets bare simple and we profit from all the lazy
>> >> >> >>>>>>> evaluation
>> >> >> >>>>>>>>>> features....
>> >> >> >>>>>>>>>>
>> >> >> >>>>>>>>>> WDYT?
>> >> >> >>>>>>>>>>
>> >> >> >>>>>>>>>>
>> >> >> >>>>>>>>>>
>> >> >> >>>>>>>>>> 2015-01-01 10:44 GMT+01:00 Anatole Tresch
>> >> >> >>>>>>> <an...@apache.org>:
>> >> >> >>>>>>>>>>
>> >> >> >>>>>>>>>>> Hi all
>> >> >> >>>>>>>>>>>
>> >> >> >>>>>>>>>>> I would like to make a *SPI proposal* for PropertyFilter
>> (on
>> >> >> >>>>>>> API side we
>> >> >> >>>>>>>>>>> have UnaryOperator<Configuration>). Hereby filters have
>> >> >> >>>>>>> ordinals and
>> >> >> >>>>>>>>>>> undergo the same ordering mechanism as PropertySources.
>> >> >> >>>>>>> Filters get
>> >> >> >>>>>>>>>>> similarly added to chain of evaluation as PropertySources
>> >> >> >>>>>>>>> (implementations
>> >> >> >>>>>>>>>>> must add @Priority -> do we need a programmatic
>> >> >> >>>>>>> getOrdinal() to have a
>> >> >> >>>>>>>>> more
>> >> >> >>>>>>>>>>> concise usae here?), with one exceptional rule:
>> >> >> >>>>>>>>>>> If a filter accidentally has the same ordinal as some
>> >> >> >>>>>>> PropertySources in
>> >> >> >>>>>>>>>>> the chain, filters are always executed AFTER
>> PropertySources
>> >> >> >>>>>>>>>>>
>> >> >> >>>>>>>>>>> So here is the interface:.
>> >> >> >>>>>>>>>>>
>> >> >> >>>>>>>>>>> /**
>> >> >> >>>>>>>>>>> * <p>Interface for filtering the current map of
>> >> >> >>>>>>> properties during the
>> >> >> >>>>>>>>>>> evaluation of the chain of PropertySources.
>> >> >> >>>>>>>>>>> * Filters can be registered using the {@link
>> >> >> >>>>>>>>>>> org.apache.tamaya.spi.ServiceContext} or added
>> >> >> >>>>>>> programmatically
>> >> >> >>>>>>>>>>> * by calling {@link
>> >> >> >>>>>>>>>>>
>> >> >> >>>>>>>
>> >> org.apache.tamaya.spi.ConfigurationContext#addPropertyFilter(int,
>> >> >> >>>>>>>>>>> PropertyFilter)}. The ordinal
>> >> >> >>>>>>>>>>> * hereby is defined by the corresponding {@code
>> >> >> >>>>>>> @Priority}
>> >> >> >>>>>>>>> annotation.</p>
>> >> >> >>>>>>>>>>> * <p>Filters </p>
>> >> >> >>>>>>>>>>> */
>> >> >> >>>>>>>>>>> public interface PropertyFilter<T>{
>> >> >> >>>>>>>>>>>
>> >> >> >>>>>>>>>>>    /**
>> >> >> >>>>>>>>>>>     * Method to check, if a filter is available. Filters
>> >> that
>> >> >> >>>>>>> are
>> >> >> >>>>>>>>>>> currently not available are not considered, when
>> >> >> >>>>>>>>>>>     * evaluating the current chain of {@link
>> >> >> >>>>>>>>>>> org.apache.tamaya.spi.PropertySource} instances.
>> >> >> >>>>>>>>>>>     * @return true, if a filter is available.
>> >> >> >>>>>>>>>>>     */
>> >> >> >>>>>>>>>>>    public boolean isAvailable();
>> >> >> >>>>>>>>>>>
>> >> >> >>>>>>>>>>>    /**
>> >> >> >>>>>>>>>>>     * Maps the current properties map to the filtered
>> >> version
>> >> >> >>>>>>> of the
>> >> >> >>>>>>>>> map.
>> >> >> >>>>>>>>>>> The resulting map will be used as the input
>> >> >> >>>>>>>>>>>     * for all subsequent handling of PropertySources in
>> the
>> >> >> >>>>>>>>>>> PropertySource chain.
>> >> >> >>>>>>>>>>>     * @param currentMap the current property map, not
>> null
>> >> >> >>>>>>>>>>>     * @return the filtered map, never null.
>> >> >> >>>>>>>>>>>     */
>> >> >> >>>>>>>>>>>    Map<String,String> map(Map<String,String>
>> >> >> >>>>>>> currentMap);
>> >> >> >>>>>>>>>>>
>> >> >> >>>>>>>>>>> }
>> >> >> >>>>>>>>>>>
>> >> >> >>>>>>>>>>> WDYT?
>> >> >> >>>>>>>>>>>
>> >> >> >>>>>>>>>>> Anatole
>> >> >> >>>>>>>>>>>
>> >> >> >>>>>>>>>>>
>> >> >> >>>>>>>>>>> 2014-12-29 17:59 GMT+01:00 Anatole Tresch
>> >> >> >>>>>>> <an...@apache.org>:
>> >> >> >>>>>>>>>>>
>> >> >> >>>>>>>>>>>> Whereas overrides are simple with this mechanism. I am
>> >> >> >>>>>>> asking how we
>> >> >> >>>>>>>>> want
>> >> >> >>>>>>>>>>>> to do other aggregation types, or filtering of values.
>> >> >> >>>>>>>>>>>>
>> >> >> >>>>>>>>>>>> *On the API side* filtering is always simple, since we
>> >> >> >>>>>>> have the
>> >> >> >>>>>>>>>>>> functional extension points
>> >> >> >>>>>>> (UnaryOperator<Configuration>), where we
>> >> >> >>>>>>>>> can
>> >> >> >>>>>>>>>>>> easily filter the whole configuration, and the adapters
>> >> >> >>>>>>> for filtering
>> >> >> >>>>>>>>>>>> single entries on access.
>> >> >> >>>>>>>>>>>>
>> >> >> >>>>>>>>>>>> But let's assume the following use case:
>> >> >> >>>>>>>>>>>>
>> >> >> >>>>>>>>>>>> *We have a system low level configuration S, which also
>> >> >> >>>>>>> contains a few
>> >> >> >>>>>>>>>>>> entries, I do not want to see on application config A.
>> >> >> >>>>>>> Given our
>> >> >> >>>>>>>>> priority
>> >> >> >>>>>>>>>>>> based assembly this is not possible, since I only can
>> >> >> >>>>>>> override the
>> >> >> >>>>>>>>> values
>> >> >> >>>>>>>>>>>> from my parent sources, but never remove them.*
>> >> >> >>>>>>>>>>>>
>> >> >> >>>>>>>>>>>> Thinking on this the following ideas come to my mind:
>> >> >> >>>>>>>>>>>>
>> >> >> >>>>>>>>>>>> 1) *add PropertySource filters to my chain. I could use
>> >> >> >>>>>>> the same
>> >> >> >>>>>>>>>>>> priorities. The bad thing is that it is not clear if the
>> >> >> >>>>>>> filter should
>> >> >> >>>>>>>>> be
>> >> >> >>>>>>>>>>>> active also, when no source with a similar priority is
>> >> >> >>>>>>> active and if
>> >> >> >>>>>>>>> so if
>> >> >> >>>>>>>>>>>> it should be before or after, so this will not work.*
>> >> >> >>>>>>>>>>>> *2) I could define a PropertySource to be filtering, so
>> it
>> >> >> >>>>>>> is able not
>> >> >> >>>>>>>>>>>> only to override, but also to remove entries. This
>> >> >> >>>>>>> "filter" can be
>> >> >> >>>>>>>>>>>> activated or deactivated depending on the current
>> context.
>> >> >> >>>>>>> Given that
>> >> >> >>>>>>>>> (if
>> >> >> >>>>>>>>>>>> we have set a correct priority value), all works fine.*
>> >> >> >>>>>>>>>>>> *3) I could define a separate filter layer which is
>> >> >> >>>>>>> allowed to filter
>> >> >> >>>>>>>>> the
>> >> >> >>>>>>>>>>>> entries, before they are published (after all overrides
>> >> >> >>>>>>> were done).
>> >> >> >>>>>>>>> This
>> >> >> >>>>>>>>>>>> would probably be transparently be hooked into the
>> >> >> >>>>>>> "Configuration
>> >> >> >>>>>>>>> Layer",
>> >> >> >>>>>>>>>>>> which takes the current property sources and creates the
>> >> >> >>>>>>> final map of
>> >> >> >>>>>>>>>>>> properties. Then on the final Map the filter service is
>> >> >> >>>>>>> operating
>> >> >> >>>>>>>>> before
>> >> >> >>>>>>>>>>>> any data is handed out to the callers.*
>> >> >> >>>>>>>>>>>>
>> >> >> >>>>>>>>>>>> I think (3) is the best solution here. Nevertheless we
>> >> >> >>>>>>> should think on
>> >> >> >>>>>>>>>>>> how this can be done the best way and how the SPI here
>> >> >> >>>>>>> would look like.
>> >> >> >>>>>>>>>>>> Also I think, we might have several filters active, e.g.
>> >> >> >>>>>>> one for
>> >> >> >>>>>>>>> logging,
>> >> >> >>>>>>>>>>>> one for admins, one which considers the current user's
>> >> >> >>>>>>> role (aka
>> >> >> >>>>>>>>> views), ...
>> >> >> >>>>>>>>>>>>
>> >> >> >>>>>>>>>>>> So ideas are welcome ;)
>> >> >> >>>>>>>>>>>>
>> >> >> >>>>>>>>>>>> Best,
>> >> >> >>>>>>>>>>>> Anatole
>> >> >> >>>>>>>>>>>>
>> >> >> >>>>>>>>>>>>
>> >> >> >>>>>>>>>>>
>> >> >> >>>>>>>>>
>> >> >> >>>>>>>>
>> >> >> >>>>>>>>
>> >> >> >>>>>>>>
>> >> >> >>>>>>>> --
>> >> >> >>>>>>>> *Anatole Tresch*
>> >> >> >>>>>>>> Java Engineer & Architect, JSR Spec Lead
>> >> >> >>>>>>>> Glärnischweg 10
>> >> >> >>>>>>>> CH - 8620 Wetzikon
>> >> >> >>>>>>>>
>> >> >> >>>>>>>> *Switzerland, Europe Zurich, GMT+1*
>> >> >> >>>>>>>> *Twitter:  @atsticks*
>> >> >> >>>>>>>> *Blogs: **http://javaremarkables.blogspot.ch/
>> >> >> >>>>>>>> <http://javaremarkables.blogspot.ch/>*
>> >> >> >>>>>>>>
>> >> >> >>>>>>>> *Google: atsticksMobile  +41-76 344 62 79*
>> >> >> >>>>>>>
>> >> >> >>>>>>
>> >> >> >>>>>
>> >> >> >>>>>
>> >> >> >>>>>
>> >> >> >>>>> --
>> >> >> >>>>> *Anatole Tresch*
>> >> >> >>>>> Java Engineer & Architect, JSR Spec Lead
>> >> >> >>>>> Glärnischweg 10
>> >> >> >>>>> CH - 8620 Wetzikon
>> >> >> >>>>>
>> >> >> >>>>> *Switzerland, Europe Zurich, GMT+1*
>> >> >> >>>>> *Twitter:  @atsticks*
>> >> >> >>>>> *Blogs: **http://javaremarkables.blogspot.ch/
>> >> >> >>>>> <http://javaremarkables.blogspot.ch/>*
>> >> >> >>>>>
>> >> >> >>>>> *Google: atsticksMobile  +41-76 344 62 79*
>> >> >> >>>>
>> >> >> >>>
>> >> >>
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > *Anatole Tresch*
>> >> > Java Engineer & Architect, JSR Spec Lead
>> >> > Glärnischweg 10
>> >> > CH - 8620 Wetzikon
>> >> >
>> >> > *Switzerland, Europe Zurich, GMT+1*
>> >> > *Twitter:  @atsticks*
>> >> > *Blogs: **http://javaremarkables.blogspot.ch/
>> >> > <http://javaremarkables.blogspot.ch/>*
>> >> >
>> >> > *Google: atsticksMobile  +41-76 344 62 79 <%2B41-76%20344%2062%2079>*
>> >>
>> >
>> >
>> >
>> > --
>> > *Anatole Tresch*
>> > Java Engineer & Architect, JSR Spec Lead
>> > Glärnischweg 10
>> > CH - 8620 Wetzikon
>> >
>> > *Switzerland, Europe Zurich, GMT+1*
>> > *Twitter:  @atsticks*
>> > *Blogs: **http://javaremarkables.blogspot.ch/
>> > <http://javaremarkables.blogspot.ch/>*
>> >
>> > *Google: atsticksMobile  +41-76 344 62 79*
>>
>
>
>
> --
> *Anatole Tresch*
> Java Engineer & Architect, JSR Spec Lead
> Glärnischweg 10
> CH - 8620 Wetzikon
>
> *Switzerland, Europe Zurich, GMT+1*
> *Twitter:  @atsticks*
> *Blogs: **http://javaremarkables.blogspot.ch/
> <http://javaremarkables.blogspot.ch/>*
>
> *Google: atsticksMobile  +41-76 344 62 79*

Re: PropertyFiltering

[Anatole Tresch <at...@gmail.com>]

It will not be part of core. But it has use cases:
e.g. you define an entry, like your locale IP-adress once under

node.ip-addr

and then reference it, instead of copying the value, e.g. by
${config:node.ip-addr}. Similar would be refs to environment and system
properties...
At the end users define, what they want. I also have heard some of them
requesting full EL support! Which is doable, but for most cases it will be
too much. But given such things are extensions each can extend Tamaya with
the feature one wants, which is one of the strengths of Tamaya, I think.


2015-01-02 10:44 GMT+01:00 Romain Manni-Bucau <rm...@gmail.com>:

> ok great - BTW not sure any placeholder is needed here
>
>
> Romain Manni-Bucau
> @rmannibucau
> http://www.tomitribe.com
> http://rmannibucau.wordpress.com
> https://github.com/rmannibucau
>
>
> 2015-01-02 10:41 GMT+01:00 Anatole Tresch <at...@gmail.com>:
> > According to you use case: when you have something like defaultValue =
> "My
> > ${config:myMappedKey} value." then, yes the filters must be applied as
> > well. But that is as well during runtime, e.g. within the Configuration
> > implementation, which consumes the things provided by the
> > ConfigurationContext. So that should work as well ;)
> >
> > 2015-01-02 10:22 GMT+01:00 Romain Manni-Bucau <rm...@gmail.com>:
> >
> >> Ok
> >>
> >> so it stays runtime.
> >>
> >> Issue I had in mind was - in pseudo code:
> >>
> >> @Config(key = "my.foo", defaultValue = "filterX:value")
> >> private String str;
> >>
> >>
> >> If my.foo is not in Configuration then filter "X" should still apply
> >> but this can only be done lazily.
> >>
> >>
> >>
> >>
> >> Romain Manni-Bucau
> >> @rmannibucau
> >> http://www.tomitribe.com
> >> http://rmannibucau.wordpress.com
> >> https://github.com/rmannibucau
> >>
> >>
> >> 2015-01-02 10:06 GMT+01:00 Anatole Tresch <at...@gmail.com>:
> >> > Hi Romain
> >> >
> >> > I try to explain again and then you can explain us what should not
> work
> >> ;)
> >> >
> >> > 1) all property sources are called to evaluate a single value, or all
> >> > properties. The ConfigurationContext provides the ordered list or
> >> property
> >> > sources. Ordering is based on ordinals. So this is the same as with
> >> > Deltaspike and will work. PropertySourceProvider allow to dynamically
> >> > register propertysources by default.
> >> > 2) the value evaluated finally may be go through the filter, e.g. for
> >> > resolving placeholders (will be an extension module). Or whatever is
> >> > appropriate.
> >> >
> >> > All other kind of filtering will happen on user level. E.g.
> categories,
> >> > metadata and similar stuff can be added as separate modules on top of
> >> that.
> >> >
> >> > Looking forward to your thaughts ;)
> >> >
> >> > Anatole
> >> >
> >> >
> >> >
> >> > 2015-01-02 9:48 GMT+01:00 Romain Manni-Bucau <rm...@gmail.com>:
> >> >
> >> >> Hi Anatole,
> >> >>
> >> >> just cause I'm not sure I got it: "end of the chain" = ?
> >> >>
> >> >> If that's after property sources as mentionned earlier it sadly
> >> >> doesn't work and you still need them at runtime.
> >> >>
> >> >>
> >> >> Romain Manni-Bucau
> >> >> @rmannibucau
> >> >> http://www.tomitribe.com
> >> >> http://rmannibucau.wordpress.com
> >> >> https://github.com/rmannibucau
> >> >>
> >> >>
> >> >> 2015-01-02 8:13 GMT+01:00 Anatole Tresch <at...@gmail.com>:
> >> >> > Hi Romain
> >> >> >
> >> >> > Nobody. (Including me) wants a tree solution ;) be it, we
> discussed a
> >> >> few things yesterday in the hangout. The only things that change,
> will
> >> be
> >> >> that you get filters at the end of the chain.
> >> >> >
> >> >> > Additionally I will add extension modules for metadata / categories
> >> >> later. But this has no effect on the core api....
> >> >> >
> >> >> > Anatole
> >> >> >
> >> >> > -
> >> >> > Anatole Tresch
> >> >> > Glärnischweg 10
> >> >> > 8620 Wetzikon
> >> >> > Tel +41 (43) 317 05 30
> >> >> > -
> >> >> > Send from Mobile
> >> >> >
> >> >> >> Am 01.01.2015 um 20:42 schrieb Romain Manni-Bucau <
> >> >> rmannibucau@gmail.com>:
> >> >> >>
> >> >> >> I wouldnt start on a tree solution and i dont agree. Aggregation
> is
> >> fine
> >> >> >> and the most performant solution, only issue is not scannable
> sources
> >> >> and
> >> >> >> we can say we dont support them IMO.
> >> >> >> Le 1 janv. 2015 20:22, "Anatole Tresch" <at...@gmail.com> a
> >> écrit :
> >> >> >>
> >> >> >>> Hi Romain
> >> >> >>>
> >> >> >>> Not necessarily, but you simply do mot want to see everything
> >> >> everywhere.
> >> >> >>> You config needs on domain level are different than on app level.
> >> You
> >> >> >>> simply do not want to see sll the low level grap. Since we do not
> >> have
> >> >> >>> named configurations anymore, we need true filtering at the
> levels
> >> spi
> >> >> >>> implementors want it to have.
> >> >> >>> Furthernore: evaluating a value with the current override
> mechanism
> >> is
> >> >> too
> >> >> >>> costly becausr you are NOT using your chain in an efficient way:
> >> when
> >> >> you
> >> >> >>> have 2 sources a and b. And b defines a value, you always have to
> >> >> evaluate
> >> >> >>> b, though the fact that b overrides a, so is not relevant for
> that
> >> >> case.
> >> >> >>> Similarly evaluating a non scannable value from a is easy. Just b
> >> must
> >> >> now,
> >> >> >>> which values are not cacheable. That coulb be done by some meta
> >> data,
> >> >> ie
> >> >> >>> with another entry compliant with some metadata scheme.
> >> >> >>>
> >> >> >>> So you get everything: efficiency , filtering, caching,
> >> >> contextuality...
> >> >> >>> The only thing that this works is to provide access to the
> current
> >> >> >>> evaluation chain eg some evaluation context:
> >> >> >>> Something like:
> >> >> >>>
> >> >> >>> Interface evslctx{
> >> >> >>>
> >> >> >>> Map<string,string> getCurrentProperties();
> >> >> >>> String getCurrentValue();
> >> >> >>> Boolean contextHasChanged();
> >> >> >>> }
> >> >> >>>
> >> >> >>> Hereby context has changed can be used to flag a change in the
> >> >> preceedong
> >> >> >>> sources , so the current source can react on the change, eg by
> >> >> reevaluating
> >> >> >>> getCurrentProperties();
> >> >> >>>
> >> >> >>> Within the ConfigContext we then can access the topmost
> >> propertysource
> >> >> >>> available (therefore available and empty are not the same!). On
> >> first
> >> >> >>> sccess it will go down the gull chsin, whereas later this might
> >> pnly be
> >> >> >>> necessary when needed...
> >> >> >>>
> >> >> >>> Anatole
> >> >> >>>
> >> >> >>> Romain Manni-Bucau <rm...@gmail.com> schrieb am Do., 1.
> Jan.
> >> >> 2015 um
> >> >> >>> 19:46:
> >> >> >>>
> >> >> >>>> Do you mean we loose the aggregation? Sounds like loosing the
> >> >> >>>> configuration management/overview then, are you sure it is good?
> >> >> >>>>
> >> >> >>>> About filtering I would just care about modifying filters and
> >> ignore
> >> >> >>>> logging ones = tamaya shouldnt log any value (can log its source
> >> if we
> >> >> >>>> need) IMO. This logging layer is a very particular need and each
> >> app
> >> >> >>>> know if a value can be logged or not so adding an abstraction
> >> which is
> >> >> >>>> not needed in practise seems like overcomplicated to me.
> >> >> >>>>
> >> >> >>>>
> >> >> >>>> Romain Manni-Bucau
> >> >> >>>> @rmannibucau
> >> >> >>>> http://www.tomitribe.com
> >> >> >>>> http://rmannibucau.wordpress.com
> >> >> >>>> https://github.com/rmannibucau
> >> >> >>>>
> >> >> >>>>
> >> >> >>>> 2015-01-01 14:18 GMT+01:00 Anatole Tresch <at...@gmail.com>:
> >> >> >>>>> See inline...
> >> >> >>>>>
> >> >> >>>>> 2015-01-01 12:57 GMT+01:00 Mark Struberg <st...@yahoo.de>:
> >> >> >>>>>
> >> >> >>>>>> If lazy or eager is a matter of each specific Filter. There
> is no
> >> >> >>> common
> >> >> >>>>>> answer to that.
> >> >> >>>>>>
> >> >> >>>>>> The only reason why we have a problem with it is that the the
> >> >> proposed
> >> >> >>>>>> interface handles the whole map.
> >> >> >>>>>
> >> >> >>>>> If we would instead just apply the filtering chain on single
> >> >> attributes
> >> >> >>>>>> (while they get queried) then all those problems would vanish.
> >> >> >>>>>>
> >> >> >>>>>
> >> >> >>>>> Given what you said above. its an implementation aspect
> >> >> >>>>> of the filter
> >> >> >>>>> , if a filter passes on a key accessed to the chain for
> >> reevaluation,
> >> >> >>>>> whereas caching others we have exactly that selective filtering
> >> >> >>>>> ... so having control where a filter is placed within the
> chain of
> >> >> >>>>> property sources for me is the crucial aspect. I also think for
> >> the
> >> >> >>> sake
> >> >> >>>> of
> >> >> >>>>> simplicity filters should have the same granularity as property
> >> >> >>> sources,
> >> >> >>>>> e.g.
> >> >> >>>>>
> >> >> >>>>> envprops -> sysprops -> domain-cfg -> domainFilter -> app-cfg
> ->
> >> >> >>>>> securityFilter
> >> >> >>>>>
> >> >> >>>>> Thats the reason why I started to think to model property
> source
> >> and
> >> >> >>>>> filters very similarly, or even model the property source as
> >> property
> >> >> >>>> chain
> >> >> >>>>> member mapping the current input properties to the output
> >> properties
> >> >> to
> >> >> >>>>> pass to the next member (in that case PropertySource may also
> >> >> undergo a
> >> >> >>>>> renaming).
> >> >> >>>>> Alternatively we could model the property chain also the other
> way
> >> >> >>> round.
> >> >> >>>>> Instead of aggregating all up until we end up with a result, we
> >> may
> >> >> >>>> simply
> >> >> >>>>> ask the highest available (therefore you need an availability
> >> flag)
> >> >> >>>>> PropertySource for the current configuration. This
> implementation
> >> >> then
> >> >> >>>> can
> >> >> >>>>> deliberately decide, if it can provide a key, or it has it to
> >> >> delegate
> >> >> >>> to
> >> >> >>>>> its parent PropertySource:
> >> >> >>>>>
> >> >> >>>>> Map<String,String> getProperties(PropertySource
> >> >> parentPropertySource);
> >> >> >>>>> Optional<String> get(String key,
> >> >> >>>>> PropertySource parentPropertySource);
> >> >> >>>>>
> >> >> >>>>> This would solve all: implementation, filtering, caching and
> still
> >> >> the
> >> >> >>>>> ConfigurationContext
> >> >> >>>>> around has control over the current chain (he passes the parent
> >> >> >>> source)..
> >> >> >>>>>
> >> >> >>>>> There is btw also a difference between filtering to change
> values
> >> >> (e.g.
> >> >> >>>>>> decrypt ciphered passwords) and filtering for the purpose of
> >> >> logging.
> >> >> >>>>>>
> >> >> >>>>> Agreed. I think should try to list/discuss the use cases in
> more
> >> >> >>> detail,
> >> >> >>>>> when and how configuration should be logged. In most cases I
> would
> >> >> >>> assume
> >> >> >>>>> that we can filter by just adding a top level filter, before
> >> passing
> >> >> >>> the
> >> >> >>>>> config to the logger...
> >> >> >>>>>
> >> >> >>>>>
> >> >> >>>>>
> >> >> >>>>>>
> >> >> >>>>>> LieGrue,
> >> >> >>>>>> strub
> >> >> >>>>>>
> >> >> >>>>>>
> >> >> >>>>>>
> >> >> >>>>>>
> >> >> >>>>>>
> >> >> >>>>>>
> >> >> >>>>>>> On Thursday, 1 January 2015, 11:25, Romain Manni-Bucau <
> >> >> >>>>>> rmannibucau@gmail.com> wrote:
> >> >> >>>>>>>> I think we need both but eager initialization should be the
> >> >> >>> default
> >> >> >>>>>>> (isScannable = true). All other usages (isScannable = false)
> >> should
> >> >> >>> be
> >> >> >>>>>>> lazy and used with caution (main case I have in mind is JNDI
> >> and I
> >> >> >>>>>>> wouldnt rely on it really today).
> >> >> >>>>>>>
> >> >> >>>>>>> What I did in my common impl of conf was to build a
> Map<String,
> >> >> >>>>>>> String> then "sources" were able to trigger a refresh event
> >> making
> >> >> >>>>>>> the
> >> >> >>>>>>> configuration rebuilding the Map then do a diff and switch if
> >> >> >>>>>>> different. This implies only eager eval and makes conf
> >> management
> >> >> >>> very
> >> >> >>>>>>> easy but still allowing updates. Only case it doesnt handle
> is
> >> lazy
> >> >> >>>>>>> eval (JNDI) but this was intended cause harder to
> >> >> >>>>>>> manage/control/monitor - view would be partial and then
> wrong if
> >> >> >>>>>>> allowed.
> >> >> >>>>>>>
> >> >> >>>>>>>
> >> >> >>>>>>>
> >> >> >>>>>>> Romain Manni-Bucau
> >> >> >>>>>>> @rmannibucau
> >> >> >>>>>>> http://www.tomitribe.com
> >> >> >>>>>>> http://rmannibucau.wordpress.com
> >> >> >>>>>>> https://github.com/rmannibucau
> >> >> >>>>>>>
> >> >> >>>>>>>
> >> >> >>>>>>>
> >> >> >>>>>>> 2015-01-01 11:18 GMT+01:00 Anatole Tresch <
> atsticks@gmail.com>:
> >> >> >>>>>>>> I would say lazyness for evaluation is a must (due to
> possible
> >> >> >>>>>> environment
> >> >> >>>>>>>> dependencies properties returned can be different!),
> caching of
> >> >> >>>>>> aggregated
> >> >> >>>>>>>> configuration on some level comes on top. Or did I get
> >> something
> >> >> >>>>>> wrong...?
> >> >> >>>>>>>>
> >> >> >>>>>>>>
> >> >> >>>>>>>> 2015-01-01 11:11 GMT+01:00 Romain Manni-Bucau
> >> >> >>>>>>> <rm...@gmail.com>:
> >> >> >>>>>>>>
> >> >> >>>>>>>>> wouldnt it make it harder to implement - ie not user
> friendly
> >> but
> >> >> >>>> impl
> >> >> >>>>>>>>> friendly only?
> >> >> >>>>>>>>>
> >> >> >>>>>>>>> + not sure lazyness is useful we need an aggregated config
> >> anyway
> >> >> >>>> (=
> >> >> >>>>>>> eager)
> >> >> >>>>>>>>>
> >> >> >>>>>>>>>
> >> >> >>>>>>>>> Romain Manni-Bucau
> >> >> >>>>>>>>> @rmannibucau
> >> >> >>>>>>>>> http://www.tomitribe.com
> >> >> >>>>>>>>> http://rmannibucau.wordpress.com
> >> >> >>>>>>>>> https://github.com/rmannibucau
> >> >> >>>>>>>>>
> >> >> >>>>>>>>>
> >> >> >>>>>>>>> 2015-01-01 11:08 GMT+01:00 Anatole Tresch <
> anatole@apache.org
> >> >:
> >> >> >>>>>>>>>> Hi all
> >> >> >>>>>>>>>>
> >> >> >>>>>>>>>> another alternative that comes to my mind, would be to
> model
> >> >> >>> the
> >> >> >>>>>>>>> PropertySource
> >> >> >>>>>>>>>> more as a Stream<Map.Entry<String,String>> than a
> >> >> >>>>>>> provider:
> >> >> >>>>>>>>>>
> >> >> >>>>>>>>>>   /**
> >> >> >>>>>>>>>>     * <p>Evaluate the current properties as Map, based on
> >> >> >>>>>>> the current
> >> >> >>>>>>>>>> defined properties. The resulting Map may not        *
> return
> >> >> >>> all
> >> >> >>>>>>> items
> >> >> >>>>>>>>>> accessible, when the PropertySource is filtering values
> for
> >> the
> >> >> >>>>>>> context,
> >> >> >>>>>>>>>> where the
> >> >> >>>>>>>>>>     * PropertySource is available.</p>
> >> >> >>>>>>>>>>     * <p>PropertySources that are not filtering data,
> >> >> >>>>>>> should either add
> >> >> >>>>>>>>> or
> >> >> >>>>>>>>>> override entries, but not remove
> >> >> >>>>>>>>>>     * any entries provided by the input map.
> >> >> >>>>>>>>>>     *
> >> >> >>>>>>>>>>     * @return the a corresponding map, never null.
> >> >> >>>>>>>>>>     * //X TODO or should we just do getPropertyKeys()?
> Think
> >> >> >>>>>>> about
> >> >> >>>>>>>>>> security (key) vs easier merging (full map)?
> >> >> >>>>>>>>>>     */
> >> >> >>>>>>>>>>    Stream<Map.Entry<String,String>> getProperties(
> >> >> >>>>>>>>>> Stream<Map.Entry<String,String>>
> >> >> >>>>>>> currentEntriesStream);
> >> >> >>>>>>>>>>
> >> >> >>>>>>>>>> This allows us to export all collecting and filtering
> >> >> >>> activities
> >> >> >>>>>>> into the
> >> >> >>>>>>>>>> Stream API, and also adding entries is simple:
> >> >> >>>>>>>>>>
> >> >> >>>>>>>>>> Stream stream = Stream.concat(stream1, stream2);
> >> >> >>>>>>>>>>
> >> >> >>>>>>>>>> Our code gets bare simple and we profit from all the lazy
> >> >> >>>>>>> evaluation
> >> >> >>>>>>>>>> features....
> >> >> >>>>>>>>>>
> >> >> >>>>>>>>>> WDYT?
> >> >> >>>>>>>>>>
> >> >> >>>>>>>>>>
> >> >> >>>>>>>>>>
> >> >> >>>>>>>>>> 2015-01-01 10:44 GMT+01:00 Anatole Tresch
> >> >> >>>>>>> <an...@apache.org>:
> >> >> >>>>>>>>>>
> >> >> >>>>>>>>>>> Hi all
> >> >> >>>>>>>>>>>
> >> >> >>>>>>>>>>> I would like to make a *SPI proposal* for PropertyFilter
> (on
> >> >> >>>>>>> API side we
> >> >> >>>>>>>>>>> have UnaryOperator<Configuration>). Hereby filters have
> >> >> >>>>>>> ordinals and
> >> >> >>>>>>>>>>> undergo the same ordering mechanism as PropertySources.
> >> >> >>>>>>> Filters get
> >> >> >>>>>>>>>>> similarly added to chain of evaluation as PropertySources
> >> >> >>>>>>>>> (implementations
> >> >> >>>>>>>>>>> must add @Priority -> do we need a programmatic
> >> >> >>>>>>> getOrdinal() to have a
> >> >> >>>>>>>>> more
> >> >> >>>>>>>>>>> concise usae here?), with one exceptional rule:
> >> >> >>>>>>>>>>> If a filter accidentally has the same ordinal as some
> >> >> >>>>>>> PropertySources in
> >> >> >>>>>>>>>>> the chain, filters are always executed AFTER
> PropertySources
> >> >> >>>>>>>>>>>
> >> >> >>>>>>>>>>> So here is the interface:.
> >> >> >>>>>>>>>>>
> >> >> >>>>>>>>>>> /**
> >> >> >>>>>>>>>>> * <p>Interface for filtering the current map of
> >> >> >>>>>>> properties during the
> >> >> >>>>>>>>>>> evaluation of the chain of PropertySources.
> >> >> >>>>>>>>>>> * Filters can be registered using the {@link
> >> >> >>>>>>>>>>> org.apache.tamaya.spi.ServiceContext} or added
> >> >> >>>>>>> programmatically
> >> >> >>>>>>>>>>> * by calling {@link
> >> >> >>>>>>>>>>>
> >> >> >>>>>>>
> >> org.apache.tamaya.spi.ConfigurationContext#addPropertyFilter(int,
> >> >> >>>>>>>>>>> PropertyFilter)}. The ordinal
> >> >> >>>>>>>>>>> * hereby is defined by the corresponding {@code
> >> >> >>>>>>> @Priority}
> >> >> >>>>>>>>> annotation.</p>
> >> >> >>>>>>>>>>> * <p>Filters </p>
> >> >> >>>>>>>>>>> */
> >> >> >>>>>>>>>>> public interface PropertyFilter<T>{
> >> >> >>>>>>>>>>>
> >> >> >>>>>>>>>>>    /**
> >> >> >>>>>>>>>>>     * Method to check, if a filter is available. Filters
> >> that
> >> >> >>>>>>> are
> >> >> >>>>>>>>>>> currently not available are not considered, when
> >> >> >>>>>>>>>>>     * evaluating the current chain of {@link
> >> >> >>>>>>>>>>> org.apache.tamaya.spi.PropertySource} instances.
> >> >> >>>>>>>>>>>     * @return true, if a filter is available.
> >> >> >>>>>>>>>>>     */
> >> >> >>>>>>>>>>>    public boolean isAvailable();
> >> >> >>>>>>>>>>>
> >> >> >>>>>>>>>>>    /**
> >> >> >>>>>>>>>>>     * Maps the current properties map to the filtered
> >> version
> >> >> >>>>>>> of the
> >> >> >>>>>>>>> map.
> >> >> >>>>>>>>>>> The resulting map will be used as the input
> >> >> >>>>>>>>>>>     * for all subsequent handling of PropertySources in
> the
> >> >> >>>>>>>>>>> PropertySource chain.
> >> >> >>>>>>>>>>>     * @param currentMap the current property map, not
> null
> >> >> >>>>>>>>>>>     * @return the filtered map, never null.
> >> >> >>>>>>>>>>>     */
> >> >> >>>>>>>>>>>    Map<String,String> map(Map<String,String>
> >> >> >>>>>>> currentMap);
> >> >> >>>>>>>>>>>
> >> >> >>>>>>>>>>> }
> >> >> >>>>>>>>>>>
> >> >> >>>>>>>>>>> WDYT?
> >> >> >>>>>>>>>>>
> >> >> >>>>>>>>>>> Anatole
> >> >> >>>>>>>>>>>
> >> >> >>>>>>>>>>>
> >> >> >>>>>>>>>>> 2014-12-29 17:59 GMT+01:00 Anatole Tresch
> >> >> >>>>>>> <an...@apache.org>:
> >> >> >>>>>>>>>>>
> >> >> >>>>>>>>>>>> Whereas overrides are simple with this mechanism. I am
> >> >> >>>>>>> asking how we
> >> >> >>>>>>>>> want
> >> >> >>>>>>>>>>>> to do other aggregation types, or filtering of values.
> >> >> >>>>>>>>>>>>
> >> >> >>>>>>>>>>>> *On the API side* filtering is always simple, since we
> >> >> >>>>>>> have the
> >> >> >>>>>>>>>>>> functional extension points
> >> >> >>>>>>> (UnaryOperator<Configuration>), where we
> >> >> >>>>>>>>> can
> >> >> >>>>>>>>>>>> easily filter the whole configuration, and the adapters
> >> >> >>>>>>> for filtering
> >> >> >>>>>>>>>>>> single entries on access.
> >> >> >>>>>>>>>>>>
> >> >> >>>>>>>>>>>> But let's assume the following use case:
> >> >> >>>>>>>>>>>>
> >> >> >>>>>>>>>>>> *We have a system low level configuration S, which also
> >> >> >>>>>>> contains a few
> >> >> >>>>>>>>>>>> entries, I do not want to see on application config A.
> >> >> >>>>>>> Given our
> >> >> >>>>>>>>> priority
> >> >> >>>>>>>>>>>> based assembly this is not possible, since I only can
> >> >> >>>>>>> override the
> >> >> >>>>>>>>> values
> >> >> >>>>>>>>>>>> from my parent sources, but never remove them.*
> >> >> >>>>>>>>>>>>
> >> >> >>>>>>>>>>>> Thinking on this the following ideas come to my mind:
> >> >> >>>>>>>>>>>>
> >> >> >>>>>>>>>>>> 1) *add PropertySource filters to my chain. I could use
> >> >> >>>>>>> the same
> >> >> >>>>>>>>>>>> priorities. The bad thing is that it is not clear if the
> >> >> >>>>>>> filter should
> >> >> >>>>>>>>> be
> >> >> >>>>>>>>>>>> active also, when no source with a similar priority is
> >> >> >>>>>>> active and if
> >> >> >>>>>>>>> so if
> >> >> >>>>>>>>>>>> it should be before or after, so this will not work.*
> >> >> >>>>>>>>>>>> *2) I could define a PropertySource to be filtering, so
> it
> >> >> >>>>>>> is able not
> >> >> >>>>>>>>>>>> only to override, but also to remove entries. This
> >> >> >>>>>>> "filter" can be
> >> >> >>>>>>>>>>>> activated or deactivated depending on the current
> context.
> >> >> >>>>>>> Given that
> >> >> >>>>>>>>> (if
> >> >> >>>>>>>>>>>> we have set a correct priority value), all works fine.*
> >> >> >>>>>>>>>>>> *3) I could define a separate filter layer which is
> >> >> >>>>>>> allowed to filter
> >> >> >>>>>>>>> the
> >> >> >>>>>>>>>>>> entries, before they are published (after all overrides
> >> >> >>>>>>> were done).
> >> >> >>>>>>>>> This
> >> >> >>>>>>>>>>>> would probably be transparently be hooked into the
> >> >> >>>>>>> "Configuration
> >> >> >>>>>>>>> Layer",
> >> >> >>>>>>>>>>>> which takes the current property sources and creates the
> >> >> >>>>>>> final map of
> >> >> >>>>>>>>>>>> properties. Then on the final Map the filter service is
> >> >> >>>>>>> operating
> >> >> >>>>>>>>> before
> >> >> >>>>>>>>>>>> any data is handed out to the callers.*
> >> >> >>>>>>>>>>>>
> >> >> >>>>>>>>>>>> I think (3) is the best solution here. Nevertheless we
> >> >> >>>>>>> should think on
> >> >> >>>>>>>>>>>> how this can be done the best way and how the SPI here
> >> >> >>>>>>> would look like.
> >> >> >>>>>>>>>>>> Also I think, we might have several filters active, e.g.
> >> >> >>>>>>> one for
> >> >> >>>>>>>>> logging,
> >> >> >>>>>>>>>>>> one for admins, one which considers the current user's
> >> >> >>>>>>> role (aka
> >> >> >>>>>>>>> views), ...
> >> >> >>>>>>>>>>>>
> >> >> >>>>>>>>>>>> So ideas are welcome ;)
> >> >> >>>>>>>>>>>>
> >> >> >>>>>>>>>>>> Best,
> >> >> >>>>>>>>>>>> Anatole
> >> >> >>>>>>>>>>>>
> >> >> >>>>>>>>>>>>
> >> >> >>>>>>>>>>>
> >> >> >>>>>>>>>
> >> >> >>>>>>>>
> >> >> >>>>>>>>
> >> >> >>>>>>>>
> >> >> >>>>>>>> --
> >> >> >>>>>>>> *Anatole Tresch*
> >> >> >>>>>>>> Java Engineer & Architect, JSR Spec Lead
> >> >> >>>>>>>> Glärnischweg 10
> >> >> >>>>>>>> CH - 8620 Wetzikon
> >> >> >>>>>>>>
> >> >> >>>>>>>> *Switzerland, Europe Zurich, GMT+1*
> >> >> >>>>>>>> *Twitter:  @atsticks*
> >> >> >>>>>>>> *Blogs: **http://javaremarkables.blogspot.ch/
> >> >> >>>>>>>> <http://javaremarkables.blogspot.ch/>*
> >> >> >>>>>>>>
> >> >> >>>>>>>> *Google: atsticksMobile  +41-76 344 62 79*
> >> >> >>>>>>>
> >> >> >>>>>>
> >> >> >>>>>
> >> >> >>>>>
> >> >> >>>>>
> >> >> >>>>> --
> >> >> >>>>> *Anatole Tresch*
> >> >> >>>>> Java Engineer & Architect, JSR Spec Lead
> >> >> >>>>> Glärnischweg 10
> >> >> >>>>> CH - 8620 Wetzikon
> >> >> >>>>>
> >> >> >>>>> *Switzerland, Europe Zurich, GMT+1*
> >> >> >>>>> *Twitter:  @atsticks*
> >> >> >>>>> *Blogs: **http://javaremarkables.blogspot.ch/
> >> >> >>>>> <http://javaremarkables.blogspot.ch/>*
> >> >> >>>>>
> >> >> >>>>> *Google: atsticksMobile  +41-76 344 62 79*
> >> >> >>>>
> >> >> >>>
> >> >>
> >> >
> >> >
> >> >
> >> > --
> >> > *Anatole Tresch*
> >> > Java Engineer & Architect, JSR Spec Lead
> >> > Glärnischweg 10
> >> > CH - 8620 Wetzikon
> >> >
> >> > *Switzerland, Europe Zurich, GMT+1*
> >> > *Twitter:  @atsticks*
> >> > *Blogs: **http://javaremarkables.blogspot.ch/
> >> > <http://javaremarkables.blogspot.ch/>*
> >> >
> >> > *Google: atsticksMobile  +41-76 344 62 79 <%2B41-76%20344%2062%2079>*
> >>
> >
> >
> >
> > --
> > *Anatole Tresch*
> > Java Engineer & Architect, JSR Spec Lead
> > Glärnischweg 10
> > CH - 8620 Wetzikon
> >
> > *Switzerland, Europe Zurich, GMT+1*
> > *Twitter:  @atsticks*
> > *Blogs: **http://javaremarkables.blogspot.ch/
> > <http://javaremarkables.blogspot.ch/>*
> >
> > *Google: atsticksMobile  +41-76 344 62 79*
>



-- 
*Anatole Tresch*
Java Engineer & Architect, JSR Spec Lead
Glärnischweg 10
CH - 8620 Wetzikon

*Switzerland, Europe Zurich, GMT+1*
*Twitter:  @atsticks*
*Blogs: **http://javaremarkables.blogspot.ch/
<http://javaremarkables.blogspot.ch/>*

*Google: atsticksMobile  +41-76 344 62 79*

Re: PropertyFiltering

[Romain Manni-Bucau <rm...@gmail.com>]

ok great - BTW not sure any placeholder is needed here


Romain Manni-Bucau
@rmannibucau
http://www.tomitribe.com
http://rmannibucau.wordpress.com
https://github.com/rmannibucau


2015-01-02 10:41 GMT+01:00 Anatole Tresch <at...@gmail.com>:
> According to you use case: when you have something like defaultValue = "My
> ${config:myMappedKey} value." then, yes the filters must be applied as
> well. But that is as well during runtime, e.g. within the Configuration
> implementation, which consumes the things provided by the
> ConfigurationContext. So that should work as well ;)
>
> 2015-01-02 10:22 GMT+01:00 Romain Manni-Bucau <rm...@gmail.com>:
>
>> Ok
>>
>> so it stays runtime.
>>
>> Issue I had in mind was - in pseudo code:
>>
>> @Config(key = "my.foo", defaultValue = "filterX:value")
>> private String str;
>>
>>
>> If my.foo is not in Configuration then filter "X" should still apply
>> but this can only be done lazily.
>>
>>
>>
>>
>> Romain Manni-Bucau
>> @rmannibucau
>> http://www.tomitribe.com
>> http://rmannibucau.wordpress.com
>> https://github.com/rmannibucau
>>
>>
>> 2015-01-02 10:06 GMT+01:00 Anatole Tresch <at...@gmail.com>:
>> > Hi Romain
>> >
>> > I try to explain again and then you can explain us what should not work
>> ;)
>> >
>> > 1) all property sources are called to evaluate a single value, or all
>> > properties. The ConfigurationContext provides the ordered list or
>> property
>> > sources. Ordering is based on ordinals. So this is the same as with
>> > Deltaspike and will work. PropertySourceProvider allow to dynamically
>> > register propertysources by default.
>> > 2) the value evaluated finally may be go through the filter, e.g. for
>> > resolving placeholders (will be an extension module). Or whatever is
>> > appropriate.
>> >
>> > All other kind of filtering will happen on user level. E.g. categories,
>> > metadata and similar stuff can be added as separate modules on top of
>> that.
>> >
>> > Looking forward to your thaughts ;)
>> >
>> > Anatole
>> >
>> >
>> >
>> > 2015-01-02 9:48 GMT+01:00 Romain Manni-Bucau <rm...@gmail.com>:
>> >
>> >> Hi Anatole,
>> >>
>> >> just cause I'm not sure I got it: "end of the chain" = ?
>> >>
>> >> If that's after property sources as mentionned earlier it sadly
>> >> doesn't work and you still need them at runtime.
>> >>
>> >>
>> >> Romain Manni-Bucau
>> >> @rmannibucau
>> >> http://www.tomitribe.com
>> >> http://rmannibucau.wordpress.com
>> >> https://github.com/rmannibucau
>> >>
>> >>
>> >> 2015-01-02 8:13 GMT+01:00 Anatole Tresch <at...@gmail.com>:
>> >> > Hi Romain
>> >> >
>> >> > Nobody. (Including me) wants a tree solution ;) be it, we discussed a
>> >> few things yesterday in the hangout. The only things that change, will
>> be
>> >> that you get filters at the end of the chain.
>> >> >
>> >> > Additionally I will add extension modules for metadata / categories
>> >> later. But this has no effect on the core api....
>> >> >
>> >> > Anatole
>> >> >
>> >> > -
>> >> > Anatole Tresch
>> >> > Glärnischweg 10
>> >> > 8620 Wetzikon
>> >> > Tel +41 (43) 317 05 30
>> >> > -
>> >> > Send from Mobile
>> >> >
>> >> >> Am 01.01.2015 um 20:42 schrieb Romain Manni-Bucau <
>> >> rmannibucau@gmail.com>:
>> >> >>
>> >> >> I wouldnt start on a tree solution and i dont agree. Aggregation is
>> fine
>> >> >> and the most performant solution, only issue is not scannable sources
>> >> and
>> >> >> we can say we dont support them IMO.
>> >> >> Le 1 janv. 2015 20:22, "Anatole Tresch" <at...@gmail.com> a
>> écrit :
>> >> >>
>> >> >>> Hi Romain
>> >> >>>
>> >> >>> Not necessarily, but you simply do mot want to see everything
>> >> everywhere.
>> >> >>> You config needs on domain level are different than on app level.
>> You
>> >> >>> simply do not want to see sll the low level grap. Since we do not
>> have
>> >> >>> named configurations anymore, we need true filtering at the levels
>> spi
>> >> >>> implementors want it to have.
>> >> >>> Furthernore: evaluating a value with the current override mechanism
>> is
>> >> too
>> >> >>> costly becausr you are NOT using your chain in an efficient way:
>> when
>> >> you
>> >> >>> have 2 sources a and b. And b defines a value, you always have to
>> >> evaluate
>> >> >>> b, though the fact that b overrides a, so is not relevant for that
>> >> case.
>> >> >>> Similarly evaluating a non scannable value from a is easy. Just b
>> must
>> >> now,
>> >> >>> which values are not cacheable. That coulb be done by some meta
>> data,
>> >> ie
>> >> >>> with another entry compliant with some metadata scheme.
>> >> >>>
>> >> >>> So you get everything: efficiency , filtering, caching,
>> >> contextuality...
>> >> >>> The only thing that this works is to provide access to the current
>> >> >>> evaluation chain eg some evaluation context:
>> >> >>> Something like:
>> >> >>>
>> >> >>> Interface evslctx{
>> >> >>>
>> >> >>> Map<string,string> getCurrentProperties();
>> >> >>> String getCurrentValue();
>> >> >>> Boolean contextHasChanged();
>> >> >>> }
>> >> >>>
>> >> >>> Hereby context has changed can be used to flag a change in the
>> >> preceedong
>> >> >>> sources , so the current source can react on the change, eg by
>> >> reevaluating
>> >> >>> getCurrentProperties();
>> >> >>>
>> >> >>> Within the ConfigContext we then can access the topmost
>> propertysource
>> >> >>> available (therefore available and empty are not the same!). On
>> first
>> >> >>> sccess it will go down the gull chsin, whereas later this might
>> pnly be
>> >> >>> necessary when needed...
>> >> >>>
>> >> >>> Anatole
>> >> >>>
>> >> >>> Romain Manni-Bucau <rm...@gmail.com> schrieb am Do., 1. Jan.
>> >> 2015 um
>> >> >>> 19:46:
>> >> >>>
>> >> >>>> Do you mean we loose the aggregation? Sounds like loosing the
>> >> >>>> configuration management/overview then, are you sure it is good?
>> >> >>>>
>> >> >>>> About filtering I would just care about modifying filters and
>> ignore
>> >> >>>> logging ones = tamaya shouldnt log any value (can log its source
>> if we
>> >> >>>> need) IMO. This logging layer is a very particular need and each
>> app
>> >> >>>> know if a value can be logged or not so adding an abstraction
>> which is
>> >> >>>> not needed in practise seems like overcomplicated to me.
>> >> >>>>
>> >> >>>>
>> >> >>>> Romain Manni-Bucau
>> >> >>>> @rmannibucau
>> >> >>>> http://www.tomitribe.com
>> >> >>>> http://rmannibucau.wordpress.com
>> >> >>>> https://github.com/rmannibucau
>> >> >>>>
>> >> >>>>
>> >> >>>> 2015-01-01 14:18 GMT+01:00 Anatole Tresch <at...@gmail.com>:
>> >> >>>>> See inline...
>> >> >>>>>
>> >> >>>>> 2015-01-01 12:57 GMT+01:00 Mark Struberg <st...@yahoo.de>:
>> >> >>>>>
>> >> >>>>>> If lazy or eager is a matter of each specific Filter. There is no
>> >> >>> common
>> >> >>>>>> answer to that.
>> >> >>>>>>
>> >> >>>>>> The only reason why we have a problem with it is that the the
>> >> proposed
>> >> >>>>>> interface handles the whole map.
>> >> >>>>>
>> >> >>>>> If we would instead just apply the filtering chain on single
>> >> attributes
>> >> >>>>>> (while they get queried) then all those problems would vanish.
>> >> >>>>>>
>> >> >>>>>
>> >> >>>>> Given what you said above. its an implementation aspect
>> >> >>>>> of the filter
>> >> >>>>> , if a filter passes on a key accessed to the chain for
>> reevaluation,
>> >> >>>>> whereas caching others we have exactly that selective filtering
>> >> >>>>> ... so having control where a filter is placed within the chain of
>> >> >>>>> property sources for me is the crucial aspect. I also think for
>> the
>> >> >>> sake
>> >> >>>> of
>> >> >>>>> simplicity filters should have the same granularity as property
>> >> >>> sources,
>> >> >>>>> e.g.
>> >> >>>>>
>> >> >>>>> envprops -> sysprops -> domain-cfg -> domainFilter -> app-cfg ->
>> >> >>>>> securityFilter
>> >> >>>>>
>> >> >>>>> Thats the reason why I started to think to model property source
>> and
>> >> >>>>> filters very similarly, or even model the property source as
>> property
>> >> >>>> chain
>> >> >>>>> member mapping the current input properties to the output
>> properties
>> >> to
>> >> >>>>> pass to the next member (in that case PropertySource may also
>> >> undergo a
>> >> >>>>> renaming).
>> >> >>>>> Alternatively we could model the property chain also the other way
>> >> >>> round.
>> >> >>>>> Instead of aggregating all up until we end up with a result, we
>> may
>> >> >>>> simply
>> >> >>>>> ask the highest available (therefore you need an availability
>> flag)
>> >> >>>>> PropertySource for the current configuration. This implementation
>> >> then
>> >> >>>> can
>> >> >>>>> deliberately decide, if it can provide a key, or it has it to
>> >> delegate
>> >> >>> to
>> >> >>>>> its parent PropertySource:
>> >> >>>>>
>> >> >>>>> Map<String,String> getProperties(PropertySource
>> >> parentPropertySource);
>> >> >>>>> Optional<String> get(String key,
>> >> >>>>> PropertySource parentPropertySource);
>> >> >>>>>
>> >> >>>>> This would solve all: implementation, filtering, caching and still
>> >> the
>> >> >>>>> ConfigurationContext
>> >> >>>>> around has control over the current chain (he passes the parent
>> >> >>> source)..
>> >> >>>>>
>> >> >>>>> There is btw also a difference between filtering to change values
>> >> (e.g.
>> >> >>>>>> decrypt ciphered passwords) and filtering for the purpose of
>> >> logging.
>> >> >>>>>>
>> >> >>>>> Agreed. I think should try to list/discuss the use cases in more
>> >> >>> detail,
>> >> >>>>> when and how configuration should be logged. In most cases I would
>> >> >>> assume
>> >> >>>>> that we can filter by just adding a top level filter, before
>> passing
>> >> >>> the
>> >> >>>>> config to the logger...
>> >> >>>>>
>> >> >>>>>
>> >> >>>>>
>> >> >>>>>>
>> >> >>>>>> LieGrue,
>> >> >>>>>> strub
>> >> >>>>>>
>> >> >>>>>>
>> >> >>>>>>
>> >> >>>>>>
>> >> >>>>>>
>> >> >>>>>>
>> >> >>>>>>> On Thursday, 1 January 2015, 11:25, Romain Manni-Bucau <
>> >> >>>>>> rmannibucau@gmail.com> wrote:
>> >> >>>>>>>> I think we need both but eager initialization should be the
>> >> >>> default
>> >> >>>>>>> (isScannable = true). All other usages (isScannable = false)
>> should
>> >> >>> be
>> >> >>>>>>> lazy and used with caution (main case I have in mind is JNDI
>> and I
>> >> >>>>>>> wouldnt rely on it really today).
>> >> >>>>>>>
>> >> >>>>>>> What I did in my common impl of conf was to build a Map<String,
>> >> >>>>>>> String> then "sources" were able to trigger a refresh event
>> making
>> >> >>>>>>> the
>> >> >>>>>>> configuration rebuilding the Map then do a diff and switch if
>> >> >>>>>>> different. This implies only eager eval and makes conf
>> management
>> >> >>> very
>> >> >>>>>>> easy but still allowing updates. Only case it doesnt handle is
>> lazy
>> >> >>>>>>> eval (JNDI) but this was intended cause harder to
>> >> >>>>>>> manage/control/monitor - view would be partial and then wrong if
>> >> >>>>>>> allowed.
>> >> >>>>>>>
>> >> >>>>>>>
>> >> >>>>>>>
>> >> >>>>>>> Romain Manni-Bucau
>> >> >>>>>>> @rmannibucau
>> >> >>>>>>> http://www.tomitribe.com
>> >> >>>>>>> http://rmannibucau.wordpress.com
>> >> >>>>>>> https://github.com/rmannibucau
>> >> >>>>>>>
>> >> >>>>>>>
>> >> >>>>>>>
>> >> >>>>>>> 2015-01-01 11:18 GMT+01:00 Anatole Tresch <at...@gmail.com>:
>> >> >>>>>>>> I would say lazyness for evaluation is a must (due to possible
>> >> >>>>>> environment
>> >> >>>>>>>> dependencies properties returned can be different!), caching of
>> >> >>>>>> aggregated
>> >> >>>>>>>> configuration on some level comes on top. Or did I get
>> something
>> >> >>>>>> wrong...?
>> >> >>>>>>>>
>> >> >>>>>>>>
>> >> >>>>>>>> 2015-01-01 11:11 GMT+01:00 Romain Manni-Bucau
>> >> >>>>>>> <rm...@gmail.com>:
>> >> >>>>>>>>
>> >> >>>>>>>>> wouldnt it make it harder to implement - ie not user friendly
>> but
>> >> >>>> impl
>> >> >>>>>>>>> friendly only?
>> >> >>>>>>>>>
>> >> >>>>>>>>> + not sure lazyness is useful we need an aggregated config
>> anyway
>> >> >>>> (=
>> >> >>>>>>> eager)
>> >> >>>>>>>>>
>> >> >>>>>>>>>
>> >> >>>>>>>>> Romain Manni-Bucau
>> >> >>>>>>>>> @rmannibucau
>> >> >>>>>>>>> http://www.tomitribe.com
>> >> >>>>>>>>> http://rmannibucau.wordpress.com
>> >> >>>>>>>>> https://github.com/rmannibucau
>> >> >>>>>>>>>
>> >> >>>>>>>>>
>> >> >>>>>>>>> 2015-01-01 11:08 GMT+01:00 Anatole Tresch <anatole@apache.org
>> >:
>> >> >>>>>>>>>> Hi all
>> >> >>>>>>>>>>
>> >> >>>>>>>>>> another alternative that comes to my mind, would be to model
>> >> >>> the
>> >> >>>>>>>>> PropertySource
>> >> >>>>>>>>>> more as a Stream<Map.Entry<String,String>> than a
>> >> >>>>>>> provider:
>> >> >>>>>>>>>>
>> >> >>>>>>>>>>   /**
>> >> >>>>>>>>>>     * <p>Evaluate the current properties as Map, based on
>> >> >>>>>>> the current
>> >> >>>>>>>>>> defined properties. The resulting Map may not        * return
>> >> >>> all
>> >> >>>>>>> items
>> >> >>>>>>>>>> accessible, when the PropertySource is filtering values for
>> the
>> >> >>>>>>> context,
>> >> >>>>>>>>>> where the
>> >> >>>>>>>>>>     * PropertySource is available.</p>
>> >> >>>>>>>>>>     * <p>PropertySources that are not filtering data,
>> >> >>>>>>> should either add
>> >> >>>>>>>>> or
>> >> >>>>>>>>>> override entries, but not remove
>> >> >>>>>>>>>>     * any entries provided by the input map.
>> >> >>>>>>>>>>     *
>> >> >>>>>>>>>>     * @return the a corresponding map, never null.
>> >> >>>>>>>>>>     * //X TODO or should we just do getPropertyKeys()? Think
>> >> >>>>>>> about
>> >> >>>>>>>>>> security (key) vs easier merging (full map)?
>> >> >>>>>>>>>>     */
>> >> >>>>>>>>>>    Stream<Map.Entry<String,String>> getProperties(
>> >> >>>>>>>>>> Stream<Map.Entry<String,String>>
>> >> >>>>>>> currentEntriesStream);
>> >> >>>>>>>>>>
>> >> >>>>>>>>>> This allows us to export all collecting and filtering
>> >> >>> activities
>> >> >>>>>>> into the
>> >> >>>>>>>>>> Stream API, and also adding entries is simple:
>> >> >>>>>>>>>>
>> >> >>>>>>>>>> Stream stream = Stream.concat(stream1, stream2);
>> >> >>>>>>>>>>
>> >> >>>>>>>>>> Our code gets bare simple and we profit from all the lazy
>> >> >>>>>>> evaluation
>> >> >>>>>>>>>> features....
>> >> >>>>>>>>>>
>> >> >>>>>>>>>> WDYT?
>> >> >>>>>>>>>>
>> >> >>>>>>>>>>
>> >> >>>>>>>>>>
>> >> >>>>>>>>>> 2015-01-01 10:44 GMT+01:00 Anatole Tresch
>> >> >>>>>>> <an...@apache.org>:
>> >> >>>>>>>>>>
>> >> >>>>>>>>>>> Hi all
>> >> >>>>>>>>>>>
>> >> >>>>>>>>>>> I would like to make a *SPI proposal* for PropertyFilter (on
>> >> >>>>>>> API side we
>> >> >>>>>>>>>>> have UnaryOperator<Configuration>). Hereby filters have
>> >> >>>>>>> ordinals and
>> >> >>>>>>>>>>> undergo the same ordering mechanism as PropertySources.
>> >> >>>>>>> Filters get
>> >> >>>>>>>>>>> similarly added to chain of evaluation as PropertySources
>> >> >>>>>>>>> (implementations
>> >> >>>>>>>>>>> must add @Priority -> do we need a programmatic
>> >> >>>>>>> getOrdinal() to have a
>> >> >>>>>>>>> more
>> >> >>>>>>>>>>> concise usae here?), with one exceptional rule:
>> >> >>>>>>>>>>> If a filter accidentally has the same ordinal as some
>> >> >>>>>>> PropertySources in
>> >> >>>>>>>>>>> the chain, filters are always executed AFTER PropertySources
>> >> >>>>>>>>>>>
>> >> >>>>>>>>>>> So here is the interface:.
>> >> >>>>>>>>>>>
>> >> >>>>>>>>>>> /**
>> >> >>>>>>>>>>> * <p>Interface for filtering the current map of
>> >> >>>>>>> properties during the
>> >> >>>>>>>>>>> evaluation of the chain of PropertySources.
>> >> >>>>>>>>>>> * Filters can be registered using the {@link
>> >> >>>>>>>>>>> org.apache.tamaya.spi.ServiceContext} or added
>> >> >>>>>>> programmatically
>> >> >>>>>>>>>>> * by calling {@link
>> >> >>>>>>>>>>>
>> >> >>>>>>>
>> org.apache.tamaya.spi.ConfigurationContext#addPropertyFilter(int,
>> >> >>>>>>>>>>> PropertyFilter)}. The ordinal
>> >> >>>>>>>>>>> * hereby is defined by the corresponding {@code
>> >> >>>>>>> @Priority}
>> >> >>>>>>>>> annotation.</p>
>> >> >>>>>>>>>>> * <p>Filters </p>
>> >> >>>>>>>>>>> */
>> >> >>>>>>>>>>> public interface PropertyFilter<T>{
>> >> >>>>>>>>>>>
>> >> >>>>>>>>>>>    /**
>> >> >>>>>>>>>>>     * Method to check, if a filter is available. Filters
>> that
>> >> >>>>>>> are
>> >> >>>>>>>>>>> currently not available are not considered, when
>> >> >>>>>>>>>>>     * evaluating the current chain of {@link
>> >> >>>>>>>>>>> org.apache.tamaya.spi.PropertySource} instances.
>> >> >>>>>>>>>>>     * @return true, if a filter is available.
>> >> >>>>>>>>>>>     */
>> >> >>>>>>>>>>>    public boolean isAvailable();
>> >> >>>>>>>>>>>
>> >> >>>>>>>>>>>    /**
>> >> >>>>>>>>>>>     * Maps the current properties map to the filtered
>> version
>> >> >>>>>>> of the
>> >> >>>>>>>>> map.
>> >> >>>>>>>>>>> The resulting map will be used as the input
>> >> >>>>>>>>>>>     * for all subsequent handling of PropertySources in the
>> >> >>>>>>>>>>> PropertySource chain.
>> >> >>>>>>>>>>>     * @param currentMap the current property map, not null
>> >> >>>>>>>>>>>     * @return the filtered map, never null.
>> >> >>>>>>>>>>>     */
>> >> >>>>>>>>>>>    Map<String,String> map(Map<String,String>
>> >> >>>>>>> currentMap);
>> >> >>>>>>>>>>>
>> >> >>>>>>>>>>> }
>> >> >>>>>>>>>>>
>> >> >>>>>>>>>>> WDYT?
>> >> >>>>>>>>>>>
>> >> >>>>>>>>>>> Anatole
>> >> >>>>>>>>>>>
>> >> >>>>>>>>>>>
>> >> >>>>>>>>>>> 2014-12-29 17:59 GMT+01:00 Anatole Tresch
>> >> >>>>>>> <an...@apache.org>:
>> >> >>>>>>>>>>>
>> >> >>>>>>>>>>>> Whereas overrides are simple with this mechanism. I am
>> >> >>>>>>> asking how we
>> >> >>>>>>>>> want
>> >> >>>>>>>>>>>> to do other aggregation types, or filtering of values.
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>> *On the API side* filtering is always simple, since we
>> >> >>>>>>> have the
>> >> >>>>>>>>>>>> functional extension points
>> >> >>>>>>> (UnaryOperator<Configuration>), where we
>> >> >>>>>>>>> can
>> >> >>>>>>>>>>>> easily filter the whole configuration, and the adapters
>> >> >>>>>>> for filtering
>> >> >>>>>>>>>>>> single entries on access.
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>> But let's assume the following use case:
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>> *We have a system low level configuration S, which also
>> >> >>>>>>> contains a few
>> >> >>>>>>>>>>>> entries, I do not want to see on application config A.
>> >> >>>>>>> Given our
>> >> >>>>>>>>> priority
>> >> >>>>>>>>>>>> based assembly this is not possible, since I only can
>> >> >>>>>>> override the
>> >> >>>>>>>>> values
>> >> >>>>>>>>>>>> from my parent sources, but never remove them.*
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>> Thinking on this the following ideas come to my mind:
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>> 1) *add PropertySource filters to my chain. I could use
>> >> >>>>>>> the same
>> >> >>>>>>>>>>>> priorities. The bad thing is that it is not clear if the
>> >> >>>>>>> filter should
>> >> >>>>>>>>> be
>> >> >>>>>>>>>>>> active also, when no source with a similar priority is
>> >> >>>>>>> active and if
>> >> >>>>>>>>> so if
>> >> >>>>>>>>>>>> it should be before or after, so this will not work.*
>> >> >>>>>>>>>>>> *2) I could define a PropertySource to be filtering, so it
>> >> >>>>>>> is able not
>> >> >>>>>>>>>>>> only to override, but also to remove entries. This
>> >> >>>>>>> "filter" can be
>> >> >>>>>>>>>>>> activated or deactivated depending on the current context.
>> >> >>>>>>> Given that
>> >> >>>>>>>>> (if
>> >> >>>>>>>>>>>> we have set a correct priority value), all works fine.*
>> >> >>>>>>>>>>>> *3) I could define a separate filter layer which is
>> >> >>>>>>> allowed to filter
>> >> >>>>>>>>> the
>> >> >>>>>>>>>>>> entries, before they are published (after all overrides
>> >> >>>>>>> were done).
>> >> >>>>>>>>> This
>> >> >>>>>>>>>>>> would probably be transparently be hooked into the
>> >> >>>>>>> "Configuration
>> >> >>>>>>>>> Layer",
>> >> >>>>>>>>>>>> which takes the current property sources and creates the
>> >> >>>>>>> final map of
>> >> >>>>>>>>>>>> properties. Then on the final Map the filter service is
>> >> >>>>>>> operating
>> >> >>>>>>>>> before
>> >> >>>>>>>>>>>> any data is handed out to the callers.*
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>> I think (3) is the best solution here. Nevertheless we
>> >> >>>>>>> should think on
>> >> >>>>>>>>>>>> how this can be done the best way and how the SPI here
>> >> >>>>>>> would look like.
>> >> >>>>>>>>>>>> Also I think, we might have several filters active, e.g.
>> >> >>>>>>> one for
>> >> >>>>>>>>> logging,
>> >> >>>>>>>>>>>> one for admins, one which considers the current user's
>> >> >>>>>>> role (aka
>> >> >>>>>>>>> views), ...
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>> So ideas are welcome ;)
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>> Best,
>> >> >>>>>>>>>>>> Anatole
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>
>> >> >>>>>>>>>
>> >> >>>>>>>>
>> >> >>>>>>>>
>> >> >>>>>>>>
>> >> >>>>>>>> --
>> >> >>>>>>>> *Anatole Tresch*
>> >> >>>>>>>> Java Engineer & Architect, JSR Spec Lead
>> >> >>>>>>>> Glärnischweg 10
>> >> >>>>>>>> CH - 8620 Wetzikon
>> >> >>>>>>>>
>> >> >>>>>>>> *Switzerland, Europe Zurich, GMT+1*
>> >> >>>>>>>> *Twitter:  @atsticks*
>> >> >>>>>>>> *Blogs: **http://javaremarkables.blogspot.ch/
>> >> >>>>>>>> <http://javaremarkables.blogspot.ch/>*
>> >> >>>>>>>>
>> >> >>>>>>>> *Google: atsticksMobile  +41-76 344 62 79*
>> >> >>>>>>>
>> >> >>>>>>
>> >> >>>>>
>> >> >>>>>
>> >> >>>>>
>> >> >>>>> --
>> >> >>>>> *Anatole Tresch*
>> >> >>>>> Java Engineer & Architect, JSR Spec Lead
>> >> >>>>> Glärnischweg 10
>> >> >>>>> CH - 8620 Wetzikon
>> >> >>>>>
>> >> >>>>> *Switzerland, Europe Zurich, GMT+1*
>> >> >>>>> *Twitter:  @atsticks*
>> >> >>>>> *Blogs: **http://javaremarkables.blogspot.ch/
>> >> >>>>> <http://javaremarkables.blogspot.ch/>*
>> >> >>>>>
>> >> >>>>> *Google: atsticksMobile  +41-76 344 62 79*
>> >> >>>>
>> >> >>>
>> >>
>> >
>> >
>> >
>> > --
>> > *Anatole Tresch*
>> > Java Engineer & Architect, JSR Spec Lead
>> > Glärnischweg 10
>> > CH - 8620 Wetzikon
>> >
>> > *Switzerland, Europe Zurich, GMT+1*
>> > *Twitter:  @atsticks*
>> > *Blogs: **http://javaremarkables.blogspot.ch/
>> > <http://javaremarkables.blogspot.ch/>*
>> >
>> > *Google: atsticksMobile  +41-76 344 62 79 <%2B41-76%20344%2062%2079>*
>>
>
>
>
> --
> *Anatole Tresch*
> Java Engineer & Architect, JSR Spec Lead
> Glärnischweg 10
> CH - 8620 Wetzikon
>
> *Switzerland, Europe Zurich, GMT+1*
> *Twitter:  @atsticks*
> *Blogs: **http://javaremarkables.blogspot.ch/
> <http://javaremarkables.blogspot.ch/>*
>
> *Google: atsticksMobile  +41-76 344 62 79*

Re: PropertyFiltering

[Anatole Tresch <at...@gmail.com>]

According to you use case: when you have something like defaultValue = "My
${config:myMappedKey} value." then, yes the filters must be applied as
well. But that is as well during runtime, e.g. within the Configuration
implementation, which consumes the things provided by the
ConfigurationContext. So that should work as well ;)

2015-01-02 10:22 GMT+01:00 Romain Manni-Bucau <rm...@gmail.com>:

> Ok
>
> so it stays runtime.
>
> Issue I had in mind was - in pseudo code:
>
> @Config(key = "my.foo", defaultValue = "filterX:value")
> private String str;
>
>
> If my.foo is not in Configuration then filter "X" should still apply
> but this can only be done lazily.
>
>
>
>
> Romain Manni-Bucau
> @rmannibucau
> http://www.tomitribe.com
> http://rmannibucau.wordpress.com
> https://github.com/rmannibucau
>
>
> 2015-01-02 10:06 GMT+01:00 Anatole Tresch <at...@gmail.com>:
> > Hi Romain
> >
> > I try to explain again and then you can explain us what should not work
> ;)
> >
> > 1) all property sources are called to evaluate a single value, or all
> > properties. The ConfigurationContext provides the ordered list or
> property
> > sources. Ordering is based on ordinals. So this is the same as with
> > Deltaspike and will work. PropertySourceProvider allow to dynamically
> > register propertysources by default.
> > 2) the value evaluated finally may be go through the filter, e.g. for
> > resolving placeholders (will be an extension module). Or whatever is
> > appropriate.
> >
> > All other kind of filtering will happen on user level. E.g. categories,
> > metadata and similar stuff can be added as separate modules on top of
> that.
> >
> > Looking forward to your thaughts ;)
> >
> > Anatole
> >
> >
> >
> > 2015-01-02 9:48 GMT+01:00 Romain Manni-Bucau <rm...@gmail.com>:
> >
> >> Hi Anatole,
> >>
> >> just cause I'm not sure I got it: "end of the chain" = ?
> >>
> >> If that's after property sources as mentionned earlier it sadly
> >> doesn't work and you still need them at runtime.
> >>
> >>
> >> Romain Manni-Bucau
> >> @rmannibucau
> >> http://www.tomitribe.com
> >> http://rmannibucau.wordpress.com
> >> https://github.com/rmannibucau
> >>
> >>
> >> 2015-01-02 8:13 GMT+01:00 Anatole Tresch <at...@gmail.com>:
> >> > Hi Romain
> >> >
> >> > Nobody. (Including me) wants a tree solution ;) be it, we discussed a
> >> few things yesterday in the hangout. The only things that change, will
> be
> >> that you get filters at the end of the chain.
> >> >
> >> > Additionally I will add extension modules for metadata / categories
> >> later. But this has no effect on the core api....
> >> >
> >> > Anatole
> >> >
> >> > -
> >> > Anatole Tresch
> >> > Glärnischweg 10
> >> > 8620 Wetzikon
> >> > Tel +41 (43) 317 05 30
> >> > -
> >> > Send from Mobile
> >> >
> >> >> Am 01.01.2015 um 20:42 schrieb Romain Manni-Bucau <
> >> rmannibucau@gmail.com>:
> >> >>
> >> >> I wouldnt start on a tree solution and i dont agree. Aggregation is
> fine
> >> >> and the most performant solution, only issue is not scannable sources
> >> and
> >> >> we can say we dont support them IMO.
> >> >> Le 1 janv. 2015 20:22, "Anatole Tresch" <at...@gmail.com> a
> écrit :
> >> >>
> >> >>> Hi Romain
> >> >>>
> >> >>> Not necessarily, but you simply do mot want to see everything
> >> everywhere.
> >> >>> You config needs on domain level are different than on app level.
> You
> >> >>> simply do not want to see sll the low level grap. Since we do not
> have
> >> >>> named configurations anymore, we need true filtering at the levels
> spi
> >> >>> implementors want it to have.
> >> >>> Furthernore: evaluating a value with the current override mechanism
> is
> >> too
> >> >>> costly becausr you are NOT using your chain in an efficient way:
> when
> >> you
> >> >>> have 2 sources a and b. And b defines a value, you always have to
> >> evaluate
> >> >>> b, though the fact that b overrides a, so is not relevant for that
> >> case.
> >> >>> Similarly evaluating a non scannable value from a is easy. Just b
> must
> >> now,
> >> >>> which values are not cacheable. That coulb be done by some meta
> data,
> >> ie
> >> >>> with another entry compliant with some metadata scheme.
> >> >>>
> >> >>> So you get everything: efficiency , filtering, caching,
> >> contextuality...
> >> >>> The only thing that this works is to provide access to the current
> >> >>> evaluation chain eg some evaluation context:
> >> >>> Something like:
> >> >>>
> >> >>> Interface evslctx{
> >> >>>
> >> >>> Map<string,string> getCurrentProperties();
> >> >>> String getCurrentValue();
> >> >>> Boolean contextHasChanged();
> >> >>> }
> >> >>>
> >> >>> Hereby context has changed can be used to flag a change in the
> >> preceedong
> >> >>> sources , so the current source can react on the change, eg by
> >> reevaluating
> >> >>> getCurrentProperties();
> >> >>>
> >> >>> Within the ConfigContext we then can access the topmost
> propertysource
> >> >>> available (therefore available and empty are not the same!). On
> first
> >> >>> sccess it will go down the gull chsin, whereas later this might
> pnly be
> >> >>> necessary when needed...
> >> >>>
> >> >>> Anatole
> >> >>>
> >> >>> Romain Manni-Bucau <rm...@gmail.com> schrieb am Do., 1. Jan.
> >> 2015 um
> >> >>> 19:46:
> >> >>>
> >> >>>> Do you mean we loose the aggregation? Sounds like loosing the
> >> >>>> configuration management/overview then, are you sure it is good?
> >> >>>>
> >> >>>> About filtering I would just care about modifying filters and
> ignore
> >> >>>> logging ones = tamaya shouldnt log any value (can log its source
> if we
> >> >>>> need) IMO. This logging layer is a very particular need and each
> app
> >> >>>> know if a value can be logged or not so adding an abstraction
> which is
> >> >>>> not needed in practise seems like overcomplicated to me.
> >> >>>>
> >> >>>>
> >> >>>> Romain Manni-Bucau
> >> >>>> @rmannibucau
> >> >>>> http://www.tomitribe.com
> >> >>>> http://rmannibucau.wordpress.com
> >> >>>> https://github.com/rmannibucau
> >> >>>>
> >> >>>>
> >> >>>> 2015-01-01 14:18 GMT+01:00 Anatole Tresch <at...@gmail.com>:
> >> >>>>> See inline...
> >> >>>>>
> >> >>>>> 2015-01-01 12:57 GMT+01:00 Mark Struberg <st...@yahoo.de>:
> >> >>>>>
> >> >>>>>> If lazy or eager is a matter of each specific Filter. There is no
> >> >>> common
> >> >>>>>> answer to that.
> >> >>>>>>
> >> >>>>>> The only reason why we have a problem with it is that the the
> >> proposed
> >> >>>>>> interface handles the whole map.
> >> >>>>>
> >> >>>>> If we would instead just apply the filtering chain on single
> >> attributes
> >> >>>>>> (while they get queried) then all those problems would vanish.
> >> >>>>>>
> >> >>>>>
> >> >>>>> Given what you said above. its an implementation aspect
> >> >>>>> of the filter
> >> >>>>> , if a filter passes on a key accessed to the chain for
> reevaluation,
> >> >>>>> whereas caching others we have exactly that selective filtering
> >> >>>>> ... so having control where a filter is placed within the chain of
> >> >>>>> property sources for me is the crucial aspect. I also think for
> the
> >> >>> sake
> >> >>>> of
> >> >>>>> simplicity filters should have the same granularity as property
> >> >>> sources,
> >> >>>>> e.g.
> >> >>>>>
> >> >>>>> envprops -> sysprops -> domain-cfg -> domainFilter -> app-cfg ->
> >> >>>>> securityFilter
> >> >>>>>
> >> >>>>> Thats the reason why I started to think to model property source
> and
> >> >>>>> filters very similarly, or even model the property source as
> property
> >> >>>> chain
> >> >>>>> member mapping the current input properties to the output
> properties
> >> to
> >> >>>>> pass to the next member (in that case PropertySource may also
> >> undergo a
> >> >>>>> renaming).
> >> >>>>> Alternatively we could model the property chain also the other way
> >> >>> round.
> >> >>>>> Instead of aggregating all up until we end up with a result, we
> may
> >> >>>> simply
> >> >>>>> ask the highest available (therefore you need an availability
> flag)
> >> >>>>> PropertySource for the current configuration. This implementation
> >> then
> >> >>>> can
> >> >>>>> deliberately decide, if it can provide a key, or it has it to
> >> delegate
> >> >>> to
> >> >>>>> its parent PropertySource:
> >> >>>>>
> >> >>>>> Map<String,String> getProperties(PropertySource
> >> parentPropertySource);
> >> >>>>> Optional<String> get(String key,
> >> >>>>> PropertySource parentPropertySource);
> >> >>>>>
> >> >>>>> This would solve all: implementation, filtering, caching and still
> >> the
> >> >>>>> ConfigurationContext
> >> >>>>> around has control over the current chain (he passes the parent
> >> >>> source)..
> >> >>>>>
> >> >>>>> There is btw also a difference between filtering to change values
> >> (e.g.
> >> >>>>>> decrypt ciphered passwords) and filtering for the purpose of
> >> logging.
> >> >>>>>>
> >> >>>>> Agreed. I think should try to list/discuss the use cases in more
> >> >>> detail,
> >> >>>>> when and how configuration should be logged. In most cases I would
> >> >>> assume
> >> >>>>> that we can filter by just adding a top level filter, before
> passing
> >> >>> the
> >> >>>>> config to the logger...
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>>>
> >> >>>>>> LieGrue,
> >> >>>>>> strub
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>> On Thursday, 1 January 2015, 11:25, Romain Manni-Bucau <
> >> >>>>>> rmannibucau@gmail.com> wrote:
> >> >>>>>>>> I think we need both but eager initialization should be the
> >> >>> default
> >> >>>>>>> (isScannable = true). All other usages (isScannable = false)
> should
> >> >>> be
> >> >>>>>>> lazy and used with caution (main case I have in mind is JNDI
> and I
> >> >>>>>>> wouldnt rely on it really today).
> >> >>>>>>>
> >> >>>>>>> What I did in my common impl of conf was to build a Map<String,
> >> >>>>>>> String> then "sources" were able to trigger a refresh event
> making
> >> >>>>>>> the
> >> >>>>>>> configuration rebuilding the Map then do a diff and switch if
> >> >>>>>>> different. This implies only eager eval and makes conf
> management
> >> >>> very
> >> >>>>>>> easy but still allowing updates. Only case it doesnt handle is
> lazy
> >> >>>>>>> eval (JNDI) but this was intended cause harder to
> >> >>>>>>> manage/control/monitor - view would be partial and then wrong if
> >> >>>>>>> allowed.
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> Romain Manni-Bucau
> >> >>>>>>> @rmannibucau
> >> >>>>>>> http://www.tomitribe.com
> >> >>>>>>> http://rmannibucau.wordpress.com
> >> >>>>>>> https://github.com/rmannibucau
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> 2015-01-01 11:18 GMT+01:00 Anatole Tresch <at...@gmail.com>:
> >> >>>>>>>> I would say lazyness for evaluation is a must (due to possible
> >> >>>>>> environment
> >> >>>>>>>> dependencies properties returned can be different!), caching of
> >> >>>>>> aggregated
> >> >>>>>>>> configuration on some level comes on top. Or did I get
> something
> >> >>>>>> wrong...?
> >> >>>>>>>>
> >> >>>>>>>>
> >> >>>>>>>> 2015-01-01 11:11 GMT+01:00 Romain Manni-Bucau
> >> >>>>>>> <rm...@gmail.com>:
> >> >>>>>>>>
> >> >>>>>>>>> wouldnt it make it harder to implement - ie not user friendly
> but
> >> >>>> impl
> >> >>>>>>>>> friendly only?
> >> >>>>>>>>>
> >> >>>>>>>>> + not sure lazyness is useful we need an aggregated config
> anyway
> >> >>>> (=
> >> >>>>>>> eager)
> >> >>>>>>>>>
> >> >>>>>>>>>
> >> >>>>>>>>> Romain Manni-Bucau
> >> >>>>>>>>> @rmannibucau
> >> >>>>>>>>> http://www.tomitribe.com
> >> >>>>>>>>> http://rmannibucau.wordpress.com
> >> >>>>>>>>> https://github.com/rmannibucau
> >> >>>>>>>>>
> >> >>>>>>>>>
> >> >>>>>>>>> 2015-01-01 11:08 GMT+01:00 Anatole Tresch <anatole@apache.org
> >:
> >> >>>>>>>>>> Hi all
> >> >>>>>>>>>>
> >> >>>>>>>>>> another alternative that comes to my mind, would be to model
> >> >>> the
> >> >>>>>>>>> PropertySource
> >> >>>>>>>>>> more as a Stream<Map.Entry<String,String>> than a
> >> >>>>>>> provider:
> >> >>>>>>>>>>
> >> >>>>>>>>>>   /**
> >> >>>>>>>>>>     * <p>Evaluate the current properties as Map, based on
> >> >>>>>>> the current
> >> >>>>>>>>>> defined properties. The resulting Map may not        * return
> >> >>> all
> >> >>>>>>> items
> >> >>>>>>>>>> accessible, when the PropertySource is filtering values for
> the
> >> >>>>>>> context,
> >> >>>>>>>>>> where the
> >> >>>>>>>>>>     * PropertySource is available.</p>
> >> >>>>>>>>>>     * <p>PropertySources that are not filtering data,
> >> >>>>>>> should either add
> >> >>>>>>>>> or
> >> >>>>>>>>>> override entries, but not remove
> >> >>>>>>>>>>     * any entries provided by the input map.
> >> >>>>>>>>>>     *
> >> >>>>>>>>>>     * @return the a corresponding map, never null.
> >> >>>>>>>>>>     * //X TODO or should we just do getPropertyKeys()? Think
> >> >>>>>>> about
> >> >>>>>>>>>> security (key) vs easier merging (full map)?
> >> >>>>>>>>>>     */
> >> >>>>>>>>>>    Stream<Map.Entry<String,String>> getProperties(
> >> >>>>>>>>>> Stream<Map.Entry<String,String>>
> >> >>>>>>> currentEntriesStream);
> >> >>>>>>>>>>
> >> >>>>>>>>>> This allows us to export all collecting and filtering
> >> >>> activities
> >> >>>>>>> into the
> >> >>>>>>>>>> Stream API, and also adding entries is simple:
> >> >>>>>>>>>>
> >> >>>>>>>>>> Stream stream = Stream.concat(stream1, stream2);
> >> >>>>>>>>>>
> >> >>>>>>>>>> Our code gets bare simple and we profit from all the lazy
> >> >>>>>>> evaluation
> >> >>>>>>>>>> features....
> >> >>>>>>>>>>
> >> >>>>>>>>>> WDYT?
> >> >>>>>>>>>>
> >> >>>>>>>>>>
> >> >>>>>>>>>>
> >> >>>>>>>>>> 2015-01-01 10:44 GMT+01:00 Anatole Tresch
> >> >>>>>>> <an...@apache.org>:
> >> >>>>>>>>>>
> >> >>>>>>>>>>> Hi all
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> I would like to make a *SPI proposal* for PropertyFilter (on
> >> >>>>>>> API side we
> >> >>>>>>>>>>> have UnaryOperator<Configuration>). Hereby filters have
> >> >>>>>>> ordinals and
> >> >>>>>>>>>>> undergo the same ordering mechanism as PropertySources.
> >> >>>>>>> Filters get
> >> >>>>>>>>>>> similarly added to chain of evaluation as PropertySources
> >> >>>>>>>>> (implementations
> >> >>>>>>>>>>> must add @Priority -> do we need a programmatic
> >> >>>>>>> getOrdinal() to have a
> >> >>>>>>>>> more
> >> >>>>>>>>>>> concise usae here?), with one exceptional rule:
> >> >>>>>>>>>>> If a filter accidentally has the same ordinal as some
> >> >>>>>>> PropertySources in
> >> >>>>>>>>>>> the chain, filters are always executed AFTER PropertySources
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> So here is the interface:.
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> /**
> >> >>>>>>>>>>> * <p>Interface for filtering the current map of
> >> >>>>>>> properties during the
> >> >>>>>>>>>>> evaluation of the chain of PropertySources.
> >> >>>>>>>>>>> * Filters can be registered using the {@link
> >> >>>>>>>>>>> org.apache.tamaya.spi.ServiceContext} or added
> >> >>>>>>> programmatically
> >> >>>>>>>>>>> * by calling {@link
> >> >>>>>>>>>>>
> >> >>>>>>>
> org.apache.tamaya.spi.ConfigurationContext#addPropertyFilter(int,
> >> >>>>>>>>>>> PropertyFilter)}. The ordinal
> >> >>>>>>>>>>> * hereby is defined by the corresponding {@code
> >> >>>>>>> @Priority}
> >> >>>>>>>>> annotation.</p>
> >> >>>>>>>>>>> * <p>Filters </p>
> >> >>>>>>>>>>> */
> >> >>>>>>>>>>> public interface PropertyFilter<T>{
> >> >>>>>>>>>>>
> >> >>>>>>>>>>>    /**
> >> >>>>>>>>>>>     * Method to check, if a filter is available. Filters
> that
> >> >>>>>>> are
> >> >>>>>>>>>>> currently not available are not considered, when
> >> >>>>>>>>>>>     * evaluating the current chain of {@link
> >> >>>>>>>>>>> org.apache.tamaya.spi.PropertySource} instances.
> >> >>>>>>>>>>>     * @return true, if a filter is available.
> >> >>>>>>>>>>>     */
> >> >>>>>>>>>>>    public boolean isAvailable();
> >> >>>>>>>>>>>
> >> >>>>>>>>>>>    /**
> >> >>>>>>>>>>>     * Maps the current properties map to the filtered
> version
> >> >>>>>>> of the
> >> >>>>>>>>> map.
> >> >>>>>>>>>>> The resulting map will be used as the input
> >> >>>>>>>>>>>     * for all subsequent handling of PropertySources in the
> >> >>>>>>>>>>> PropertySource chain.
> >> >>>>>>>>>>>     * @param currentMap the current property map, not null
> >> >>>>>>>>>>>     * @return the filtered map, never null.
> >> >>>>>>>>>>>     */
> >> >>>>>>>>>>>    Map<String,String> map(Map<String,String>
> >> >>>>>>> currentMap);
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> }
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> WDYT?
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> Anatole
> >> >>>>>>>>>>>
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> 2014-12-29 17:59 GMT+01:00 Anatole Tresch
> >> >>>>>>> <an...@apache.org>:
> >> >>>>>>>>>>>
> >> >>>>>>>>>>>> Whereas overrides are simple with this mechanism. I am
> >> >>>>>>> asking how we
> >> >>>>>>>>> want
> >> >>>>>>>>>>>> to do other aggregation types, or filtering of values.
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>> *On the API side* filtering is always simple, since we
> >> >>>>>>> have the
> >> >>>>>>>>>>>> functional extension points
> >> >>>>>>> (UnaryOperator<Configuration>), where we
> >> >>>>>>>>> can
> >> >>>>>>>>>>>> easily filter the whole configuration, and the adapters
> >> >>>>>>> for filtering
> >> >>>>>>>>>>>> single entries on access.
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>> But let's assume the following use case:
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>> *We have a system low level configuration S, which also
> >> >>>>>>> contains a few
> >> >>>>>>>>>>>> entries, I do not want to see on application config A.
> >> >>>>>>> Given our
> >> >>>>>>>>> priority
> >> >>>>>>>>>>>> based assembly this is not possible, since I only can
> >> >>>>>>> override the
> >> >>>>>>>>> values
> >> >>>>>>>>>>>> from my parent sources, but never remove them.*
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>> Thinking on this the following ideas come to my mind:
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>> 1) *add PropertySource filters to my chain. I could use
> >> >>>>>>> the same
> >> >>>>>>>>>>>> priorities. The bad thing is that it is not clear if the
> >> >>>>>>> filter should
> >> >>>>>>>>> be
> >> >>>>>>>>>>>> active also, when no source with a similar priority is
> >> >>>>>>> active and if
> >> >>>>>>>>> so if
> >> >>>>>>>>>>>> it should be before or after, so this will not work.*
> >> >>>>>>>>>>>> *2) I could define a PropertySource to be filtering, so it
> >> >>>>>>> is able not
> >> >>>>>>>>>>>> only to override, but also to remove entries. This
> >> >>>>>>> "filter" can be
> >> >>>>>>>>>>>> activated or deactivated depending on the current context.
> >> >>>>>>> Given that
> >> >>>>>>>>> (if
> >> >>>>>>>>>>>> we have set a correct priority value), all works fine.*
> >> >>>>>>>>>>>> *3) I could define a separate filter layer which is
> >> >>>>>>> allowed to filter
> >> >>>>>>>>> the
> >> >>>>>>>>>>>> entries, before they are published (after all overrides
> >> >>>>>>> were done).
> >> >>>>>>>>> This
> >> >>>>>>>>>>>> would probably be transparently be hooked into the
> >> >>>>>>> "Configuration
> >> >>>>>>>>> Layer",
> >> >>>>>>>>>>>> which takes the current property sources and creates the
> >> >>>>>>> final map of
> >> >>>>>>>>>>>> properties. Then on the final Map the filter service is
> >> >>>>>>> operating
> >> >>>>>>>>> before
> >> >>>>>>>>>>>> any data is handed out to the callers.*
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>> I think (3) is the best solution here. Nevertheless we
> >> >>>>>>> should think on
> >> >>>>>>>>>>>> how this can be done the best way and how the SPI here
> >> >>>>>>> would look like.
> >> >>>>>>>>>>>> Also I think, we might have several filters active, e.g.
> >> >>>>>>> one for
> >> >>>>>>>>> logging,
> >> >>>>>>>>>>>> one for admins, one which considers the current user's
> >> >>>>>>> role (aka
> >> >>>>>>>>> views), ...
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>> So ideas are welcome ;)
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>> Best,
> >> >>>>>>>>>>>> Anatole
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>
> >> >>>>>>>>>
> >> >>>>>>>>
> >> >>>>>>>>
> >> >>>>>>>>
> >> >>>>>>>> --
> >> >>>>>>>> *Anatole Tresch*
> >> >>>>>>>> Java Engineer & Architect, JSR Spec Lead
> >> >>>>>>>> Glärnischweg 10
> >> >>>>>>>> CH - 8620 Wetzikon
> >> >>>>>>>>
> >> >>>>>>>> *Switzerland, Europe Zurich, GMT+1*
> >> >>>>>>>> *Twitter:  @atsticks*
> >> >>>>>>>> *Blogs: **http://javaremarkables.blogspot.ch/
> >> >>>>>>>> <http://javaremarkables.blogspot.ch/>*
> >> >>>>>>>>
> >> >>>>>>>> *Google: atsticksMobile  +41-76 344 62 79*
> >> >>>>>>>
> >> >>>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>> --
> >> >>>>> *Anatole Tresch*
> >> >>>>> Java Engineer & Architect, JSR Spec Lead
> >> >>>>> Glärnischweg 10
> >> >>>>> CH - 8620 Wetzikon
> >> >>>>>
> >> >>>>> *Switzerland, Europe Zurich, GMT+1*
> >> >>>>> *Twitter:  @atsticks*
> >> >>>>> *Blogs: **http://javaremarkables.blogspot.ch/
> >> >>>>> <http://javaremarkables.blogspot.ch/>*
> >> >>>>>
> >> >>>>> *Google: atsticksMobile  +41-76 344 62 79*
> >> >>>>
> >> >>>
> >>
> >
> >
> >
> > --
> > *Anatole Tresch*
> > Java Engineer & Architect, JSR Spec Lead
> > Glärnischweg 10
> > CH - 8620 Wetzikon
> >
> > *Switzerland, Europe Zurich, GMT+1*
> > *Twitter:  @atsticks*
> > *Blogs: **http://javaremarkables.blogspot.ch/
> > <http://javaremarkables.blogspot.ch/>*
> >
> > *Google: atsticksMobile  +41-76 344 62 79 <%2B41-76%20344%2062%2079>*
>



-- 
*Anatole Tresch*
Java Engineer & Architect, JSR Spec Lead
Glärnischweg 10
CH - 8620 Wetzikon

*Switzerland, Europe Zurich, GMT+1*
*Twitter:  @atsticks*
*Blogs: **http://javaremarkables.blogspot.ch/
<http://javaremarkables.blogspot.ch/>*

*Google: atsticksMobile  +41-76 344 62 79*

Re: PropertyFiltering

[Romain Manni-Bucau <rm...@gmail.com>]

Ok

so it stays runtime.

Issue I had in mind was - in pseudo code:

@Config(key = "my.foo", defaultValue = "filterX:value")
private String str;


If my.foo is not in Configuration then filter "X" should still apply
but this can only be done lazily.




Romain Manni-Bucau
@rmannibucau
http://www.tomitribe.com
http://rmannibucau.wordpress.com
https://github.com/rmannibucau


2015-01-02 10:06 GMT+01:00 Anatole Tresch <at...@gmail.com>:
> Hi Romain
>
> I try to explain again and then you can explain us what should not work ;)
>
> 1) all property sources are called to evaluate a single value, or all
> properties. The ConfigurationContext provides the ordered list or property
> sources. Ordering is based on ordinals. So this is the same as with
> Deltaspike and will work. PropertySourceProvider allow to dynamically
> register propertysources by default.
> 2) the value evaluated finally may be go through the filter, e.g. for
> resolving placeholders (will be an extension module). Or whatever is
> appropriate.
>
> All other kind of filtering will happen on user level. E.g. categories,
> metadata and similar stuff can be added as separate modules on top of that.
>
> Looking forward to your thaughts ;)
>
> Anatole
>
>
>
> 2015-01-02 9:48 GMT+01:00 Romain Manni-Bucau <rm...@gmail.com>:
>
>> Hi Anatole,
>>
>> just cause I'm not sure I got it: "end of the chain" = ?
>>
>> If that's after property sources as mentionned earlier it sadly
>> doesn't work and you still need them at runtime.
>>
>>
>> Romain Manni-Bucau
>> @rmannibucau
>> http://www.tomitribe.com
>> http://rmannibucau.wordpress.com
>> https://github.com/rmannibucau
>>
>>
>> 2015-01-02 8:13 GMT+01:00 Anatole Tresch <at...@gmail.com>:
>> > Hi Romain
>> >
>> > Nobody. (Including me) wants a tree solution ;) be it, we discussed a
>> few things yesterday in the hangout. The only things that change, will be
>> that you get filters at the end of the chain.
>> >
>> > Additionally I will add extension modules for metadata / categories
>> later. But this has no effect on the core api....
>> >
>> > Anatole
>> >
>> > -
>> > Anatole Tresch
>> > Glärnischweg 10
>> > 8620 Wetzikon
>> > Tel +41 (43) 317 05 30
>> > -
>> > Send from Mobile
>> >
>> >> Am 01.01.2015 um 20:42 schrieb Romain Manni-Bucau <
>> rmannibucau@gmail.com>:
>> >>
>> >> I wouldnt start on a tree solution and i dont agree. Aggregation is fine
>> >> and the most performant solution, only issue is not scannable sources
>> and
>> >> we can say we dont support them IMO.
>> >> Le 1 janv. 2015 20:22, "Anatole Tresch" <at...@gmail.com> a écrit :
>> >>
>> >>> Hi Romain
>> >>>
>> >>> Not necessarily, but you simply do mot want to see everything
>> everywhere.
>> >>> You config needs on domain level are different than on app level. You
>> >>> simply do not want to see sll the low level grap. Since we do not have
>> >>> named configurations anymore, we need true filtering at the levels spi
>> >>> implementors want it to have.
>> >>> Furthernore: evaluating a value with the current override mechanism is
>> too
>> >>> costly becausr you are NOT using your chain in an efficient way: when
>> you
>> >>> have 2 sources a and b. And b defines a value, you always have to
>> evaluate
>> >>> b, though the fact that b overrides a, so is not relevant for that
>> case.
>> >>> Similarly evaluating a non scannable value from a is easy. Just b must
>> now,
>> >>> which values are not cacheable. That coulb be done by some meta data,
>> ie
>> >>> with another entry compliant with some metadata scheme.
>> >>>
>> >>> So you get everything: efficiency , filtering, caching,
>> contextuality...
>> >>> The only thing that this works is to provide access to the current
>> >>> evaluation chain eg some evaluation context:
>> >>> Something like:
>> >>>
>> >>> Interface evslctx{
>> >>>
>> >>> Map<string,string> getCurrentProperties();
>> >>> String getCurrentValue();
>> >>> Boolean contextHasChanged();
>> >>> }
>> >>>
>> >>> Hereby context has changed can be used to flag a change in the
>> preceedong
>> >>> sources , so the current source can react on the change, eg by
>> reevaluating
>> >>> getCurrentProperties();
>> >>>
>> >>> Within the ConfigContext we then can access the topmost propertysource
>> >>> available (therefore available and empty are not the same!). On first
>> >>> sccess it will go down the gull chsin, whereas later this might pnly be
>> >>> necessary when needed...
>> >>>
>> >>> Anatole
>> >>>
>> >>> Romain Manni-Bucau <rm...@gmail.com> schrieb am Do., 1. Jan.
>> 2015 um
>> >>> 19:46:
>> >>>
>> >>>> Do you mean we loose the aggregation? Sounds like loosing the
>> >>>> configuration management/overview then, are you sure it is good?
>> >>>>
>> >>>> About filtering I would just care about modifying filters and ignore
>> >>>> logging ones = tamaya shouldnt log any value (can log its source if we
>> >>>> need) IMO. This logging layer is a very particular need and each app
>> >>>> know if a value can be logged or not so adding an abstraction which is
>> >>>> not needed in practise seems like overcomplicated to me.
>> >>>>
>> >>>>
>> >>>> Romain Manni-Bucau
>> >>>> @rmannibucau
>> >>>> http://www.tomitribe.com
>> >>>> http://rmannibucau.wordpress.com
>> >>>> https://github.com/rmannibucau
>> >>>>
>> >>>>
>> >>>> 2015-01-01 14:18 GMT+01:00 Anatole Tresch <at...@gmail.com>:
>> >>>>> See inline...
>> >>>>>
>> >>>>> 2015-01-01 12:57 GMT+01:00 Mark Struberg <st...@yahoo.de>:
>> >>>>>
>> >>>>>> If lazy or eager is a matter of each specific Filter. There is no
>> >>> common
>> >>>>>> answer to that.
>> >>>>>>
>> >>>>>> The only reason why we have a problem with it is that the the
>> proposed
>> >>>>>> interface handles the whole map.
>> >>>>>
>> >>>>> If we would instead just apply the filtering chain on single
>> attributes
>> >>>>>> (while they get queried) then all those problems would vanish.
>> >>>>>>
>> >>>>>
>> >>>>> Given what you said above. its an implementation aspect
>> >>>>> of the filter
>> >>>>> , if a filter passes on a key accessed to the chain for reevaluation,
>> >>>>> whereas caching others we have exactly that selective filtering
>> >>>>> ... so having control where a filter is placed within the chain of
>> >>>>> property sources for me is the crucial aspect. I also think for the
>> >>> sake
>> >>>> of
>> >>>>> simplicity filters should have the same granularity as property
>> >>> sources,
>> >>>>> e.g.
>> >>>>>
>> >>>>> envprops -> sysprops -> domain-cfg -> domainFilter -> app-cfg ->
>> >>>>> securityFilter
>> >>>>>
>> >>>>> Thats the reason why I started to think to model property source and
>> >>>>> filters very similarly, or even model the property source as property
>> >>>> chain
>> >>>>> member mapping the current input properties to the output properties
>> to
>> >>>>> pass to the next member (in that case PropertySource may also
>> undergo a
>> >>>>> renaming).
>> >>>>> Alternatively we could model the property chain also the other way
>> >>> round.
>> >>>>> Instead of aggregating all up until we end up with a result, we may
>> >>>> simply
>> >>>>> ask the highest available (therefore you need an availability flag)
>> >>>>> PropertySource for the current configuration. This implementation
>> then
>> >>>> can
>> >>>>> deliberately decide, if it can provide a key, or it has it to
>> delegate
>> >>> to
>> >>>>> its parent PropertySource:
>> >>>>>
>> >>>>> Map<String,String> getProperties(PropertySource
>> parentPropertySource);
>> >>>>> Optional<String> get(String key,
>> >>>>> PropertySource parentPropertySource);
>> >>>>>
>> >>>>> This would solve all: implementation, filtering, caching and still
>> the
>> >>>>> ConfigurationContext
>> >>>>> around has control over the current chain (he passes the parent
>> >>> source)..
>> >>>>>
>> >>>>> There is btw also a difference between filtering to change values
>> (e.g.
>> >>>>>> decrypt ciphered passwords) and filtering for the purpose of
>> logging.
>> >>>>>>
>> >>>>> Agreed. I think should try to list/discuss the use cases in more
>> >>> detail,
>> >>>>> when and how configuration should be logged. In most cases I would
>> >>> assume
>> >>>>> that we can filter by just adding a top level filter, before passing
>> >>> the
>> >>>>> config to the logger...
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>>
>> >>>>>> LieGrue,
>> >>>>>> strub
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>> On Thursday, 1 January 2015, 11:25, Romain Manni-Bucau <
>> >>>>>> rmannibucau@gmail.com> wrote:
>> >>>>>>>> I think we need both but eager initialization should be the
>> >>> default
>> >>>>>>> (isScannable = true). All other usages (isScannable = false) should
>> >>> be
>> >>>>>>> lazy and used with caution (main case I have in mind is JNDI and I
>> >>>>>>> wouldnt rely on it really today).
>> >>>>>>>
>> >>>>>>> What I did in my common impl of conf was to build a Map<String,
>> >>>>>>> String> then "sources" were able to trigger a refresh event making
>> >>>>>>> the
>> >>>>>>> configuration rebuilding the Map then do a diff and switch if
>> >>>>>>> different. This implies only eager eval and makes conf management
>> >>> very
>> >>>>>>> easy but still allowing updates. Only case it doesnt handle is lazy
>> >>>>>>> eval (JNDI) but this was intended cause harder to
>> >>>>>>> manage/control/monitor - view would be partial and then wrong if
>> >>>>>>> allowed.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Romain Manni-Bucau
>> >>>>>>> @rmannibucau
>> >>>>>>> http://www.tomitribe.com
>> >>>>>>> http://rmannibucau.wordpress.com
>> >>>>>>> https://github.com/rmannibucau
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> 2015-01-01 11:18 GMT+01:00 Anatole Tresch <at...@gmail.com>:
>> >>>>>>>> I would say lazyness for evaluation is a must (due to possible
>> >>>>>> environment
>> >>>>>>>> dependencies properties returned can be different!), caching of
>> >>>>>> aggregated
>> >>>>>>>> configuration on some level comes on top. Or did I get something
>> >>>>>> wrong...?
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> 2015-01-01 11:11 GMT+01:00 Romain Manni-Bucau
>> >>>>>>> <rm...@gmail.com>:
>> >>>>>>>>
>> >>>>>>>>> wouldnt it make it harder to implement - ie not user friendly but
>> >>>> impl
>> >>>>>>>>> friendly only?
>> >>>>>>>>>
>> >>>>>>>>> + not sure lazyness is useful we need an aggregated config anyway
>> >>>> (=
>> >>>>>>> eager)
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> Romain Manni-Bucau
>> >>>>>>>>> @rmannibucau
>> >>>>>>>>> http://www.tomitribe.com
>> >>>>>>>>> http://rmannibucau.wordpress.com
>> >>>>>>>>> https://github.com/rmannibucau
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> 2015-01-01 11:08 GMT+01:00 Anatole Tresch <an...@apache.org>:
>> >>>>>>>>>> Hi all
>> >>>>>>>>>>
>> >>>>>>>>>> another alternative that comes to my mind, would be to model
>> >>> the
>> >>>>>>>>> PropertySource
>> >>>>>>>>>> more as a Stream<Map.Entry<String,String>> than a
>> >>>>>>> provider:
>> >>>>>>>>>>
>> >>>>>>>>>>   /**
>> >>>>>>>>>>     * <p>Evaluate the current properties as Map, based on
>> >>>>>>> the current
>> >>>>>>>>>> defined properties. The resulting Map may not        * return
>> >>> all
>> >>>>>>> items
>> >>>>>>>>>> accessible, when the PropertySource is filtering values for the
>> >>>>>>> context,
>> >>>>>>>>>> where the
>> >>>>>>>>>>     * PropertySource is available.</p>
>> >>>>>>>>>>     * <p>PropertySources that are not filtering data,
>> >>>>>>> should either add
>> >>>>>>>>> or
>> >>>>>>>>>> override entries, but not remove
>> >>>>>>>>>>     * any entries provided by the input map.
>> >>>>>>>>>>     *
>> >>>>>>>>>>     * @return the a corresponding map, never null.
>> >>>>>>>>>>     * //X TODO or should we just do getPropertyKeys()? Think
>> >>>>>>> about
>> >>>>>>>>>> security (key) vs easier merging (full map)?
>> >>>>>>>>>>     */
>> >>>>>>>>>>    Stream<Map.Entry<String,String>> getProperties(
>> >>>>>>>>>> Stream<Map.Entry<String,String>>
>> >>>>>>> currentEntriesStream);
>> >>>>>>>>>>
>> >>>>>>>>>> This allows us to export all collecting and filtering
>> >>> activities
>> >>>>>>> into the
>> >>>>>>>>>> Stream API, and also adding entries is simple:
>> >>>>>>>>>>
>> >>>>>>>>>> Stream stream = Stream.concat(stream1, stream2);
>> >>>>>>>>>>
>> >>>>>>>>>> Our code gets bare simple and we profit from all the lazy
>> >>>>>>> evaluation
>> >>>>>>>>>> features....
>> >>>>>>>>>>
>> >>>>>>>>>> WDYT?
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> 2015-01-01 10:44 GMT+01:00 Anatole Tresch
>> >>>>>>> <an...@apache.org>:
>> >>>>>>>>>>
>> >>>>>>>>>>> Hi all
>> >>>>>>>>>>>
>> >>>>>>>>>>> I would like to make a *SPI proposal* for PropertyFilter (on
>> >>>>>>> API side we
>> >>>>>>>>>>> have UnaryOperator<Configuration>). Hereby filters have
>> >>>>>>> ordinals and
>> >>>>>>>>>>> undergo the same ordering mechanism as PropertySources.
>> >>>>>>> Filters get
>> >>>>>>>>>>> similarly added to chain of evaluation as PropertySources
>> >>>>>>>>> (implementations
>> >>>>>>>>>>> must add @Priority -> do we need a programmatic
>> >>>>>>> getOrdinal() to have a
>> >>>>>>>>> more
>> >>>>>>>>>>> concise usae here?), with one exceptional rule:
>> >>>>>>>>>>> If a filter accidentally has the same ordinal as some
>> >>>>>>> PropertySources in
>> >>>>>>>>>>> the chain, filters are always executed AFTER PropertySources
>> >>>>>>>>>>>
>> >>>>>>>>>>> So here is the interface:.
>> >>>>>>>>>>>
>> >>>>>>>>>>> /**
>> >>>>>>>>>>> * <p>Interface for filtering the current map of
>> >>>>>>> properties during the
>> >>>>>>>>>>> evaluation of the chain of PropertySources.
>> >>>>>>>>>>> * Filters can be registered using the {@link
>> >>>>>>>>>>> org.apache.tamaya.spi.ServiceContext} or added
>> >>>>>>> programmatically
>> >>>>>>>>>>> * by calling {@link
>> >>>>>>>>>>>
>> >>>>>>> org.apache.tamaya.spi.ConfigurationContext#addPropertyFilter(int,
>> >>>>>>>>>>> PropertyFilter)}. The ordinal
>> >>>>>>>>>>> * hereby is defined by the corresponding {@code
>> >>>>>>> @Priority}
>> >>>>>>>>> annotation.</p>
>> >>>>>>>>>>> * <p>Filters </p>
>> >>>>>>>>>>> */
>> >>>>>>>>>>> public interface PropertyFilter<T>{
>> >>>>>>>>>>>
>> >>>>>>>>>>>    /**
>> >>>>>>>>>>>     * Method to check, if a filter is available. Filters that
>> >>>>>>> are
>> >>>>>>>>>>> currently not available are not considered, when
>> >>>>>>>>>>>     * evaluating the current chain of {@link
>> >>>>>>>>>>> org.apache.tamaya.spi.PropertySource} instances.
>> >>>>>>>>>>>     * @return true, if a filter is available.
>> >>>>>>>>>>>     */
>> >>>>>>>>>>>    public boolean isAvailable();
>> >>>>>>>>>>>
>> >>>>>>>>>>>    /**
>> >>>>>>>>>>>     * Maps the current properties map to the filtered version
>> >>>>>>> of the
>> >>>>>>>>> map.
>> >>>>>>>>>>> The resulting map will be used as the input
>> >>>>>>>>>>>     * for all subsequent handling of PropertySources in the
>> >>>>>>>>>>> PropertySource chain.
>> >>>>>>>>>>>     * @param currentMap the current property map, not null
>> >>>>>>>>>>>     * @return the filtered map, never null.
>> >>>>>>>>>>>     */
>> >>>>>>>>>>>    Map<String,String> map(Map<String,String>
>> >>>>>>> currentMap);
>> >>>>>>>>>>>
>> >>>>>>>>>>> }
>> >>>>>>>>>>>
>> >>>>>>>>>>> WDYT?
>> >>>>>>>>>>>
>> >>>>>>>>>>> Anatole
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> 2014-12-29 17:59 GMT+01:00 Anatole Tresch
>> >>>>>>> <an...@apache.org>:
>> >>>>>>>>>>>
>> >>>>>>>>>>>> Whereas overrides are simple with this mechanism. I am
>> >>>>>>> asking how we
>> >>>>>>>>> want
>> >>>>>>>>>>>> to do other aggregation types, or filtering of values.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> *On the API side* filtering is always simple, since we
>> >>>>>>> have the
>> >>>>>>>>>>>> functional extension points
>> >>>>>>> (UnaryOperator<Configuration>), where we
>> >>>>>>>>> can
>> >>>>>>>>>>>> easily filter the whole configuration, and the adapters
>> >>>>>>> for filtering
>> >>>>>>>>>>>> single entries on access.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> But let's assume the following use case:
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> *We have a system low level configuration S, which also
>> >>>>>>> contains a few
>> >>>>>>>>>>>> entries, I do not want to see on application config A.
>> >>>>>>> Given our
>> >>>>>>>>> priority
>> >>>>>>>>>>>> based assembly this is not possible, since I only can
>> >>>>>>> override the
>> >>>>>>>>> values
>> >>>>>>>>>>>> from my parent sources, but never remove them.*
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> Thinking on this the following ideas come to my mind:
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> 1) *add PropertySource filters to my chain. I could use
>> >>>>>>> the same
>> >>>>>>>>>>>> priorities. The bad thing is that it is not clear if the
>> >>>>>>> filter should
>> >>>>>>>>> be
>> >>>>>>>>>>>> active also, when no source with a similar priority is
>> >>>>>>> active and if
>> >>>>>>>>> so if
>> >>>>>>>>>>>> it should be before or after, so this will not work.*
>> >>>>>>>>>>>> *2) I could define a PropertySource to be filtering, so it
>> >>>>>>> is able not
>> >>>>>>>>>>>> only to override, but also to remove entries. This
>> >>>>>>> "filter" can be
>> >>>>>>>>>>>> activated or deactivated depending on the current context.
>> >>>>>>> Given that
>> >>>>>>>>> (if
>> >>>>>>>>>>>> we have set a correct priority value), all works fine.*
>> >>>>>>>>>>>> *3) I could define a separate filter layer which is
>> >>>>>>> allowed to filter
>> >>>>>>>>> the
>> >>>>>>>>>>>> entries, before they are published (after all overrides
>> >>>>>>> were done).
>> >>>>>>>>> This
>> >>>>>>>>>>>> would probably be transparently be hooked into the
>> >>>>>>> "Configuration
>> >>>>>>>>> Layer",
>> >>>>>>>>>>>> which takes the current property sources and creates the
>> >>>>>>> final map of
>> >>>>>>>>>>>> properties. Then on the final Map the filter service is
>> >>>>>>> operating
>> >>>>>>>>> before
>> >>>>>>>>>>>> any data is handed out to the callers.*
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> I think (3) is the best solution here. Nevertheless we
>> >>>>>>> should think on
>> >>>>>>>>>>>> how this can be done the best way and how the SPI here
>> >>>>>>> would look like.
>> >>>>>>>>>>>> Also I think, we might have several filters active, e.g.
>> >>>>>>> one for
>> >>>>>>>>> logging,
>> >>>>>>>>>>>> one for admins, one which considers the current user's
>> >>>>>>> role (aka
>> >>>>>>>>> views), ...
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> So ideas are welcome ;)
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> Best,
>> >>>>>>>>>>>> Anatole
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> --
>> >>>>>>>> *Anatole Tresch*
>> >>>>>>>> Java Engineer & Architect, JSR Spec Lead
>> >>>>>>>> Glärnischweg 10
>> >>>>>>>> CH - 8620 Wetzikon
>> >>>>>>>>
>> >>>>>>>> *Switzerland, Europe Zurich, GMT+1*
>> >>>>>>>> *Twitter:  @atsticks*
>> >>>>>>>> *Blogs: **http://javaremarkables.blogspot.ch/
>> >>>>>>>> <http://javaremarkables.blogspot.ch/>*
>> >>>>>>>>
>> >>>>>>>> *Google: atsticksMobile  +41-76 344 62 79*
>> >>>>>>>
>> >>>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> --
>> >>>>> *Anatole Tresch*
>> >>>>> Java Engineer & Architect, JSR Spec Lead
>> >>>>> Glärnischweg 10
>> >>>>> CH - 8620 Wetzikon
>> >>>>>
>> >>>>> *Switzerland, Europe Zurich, GMT+1*
>> >>>>> *Twitter:  @atsticks*
>> >>>>> *Blogs: **http://javaremarkables.blogspot.ch/
>> >>>>> <http://javaremarkables.blogspot.ch/>*
>> >>>>>
>> >>>>> *Google: atsticksMobile  +41-76 344 62 79*
>> >>>>
>> >>>
>>
>
>
>
> --
> *Anatole Tresch*
> Java Engineer & Architect, JSR Spec Lead
> Glärnischweg 10
> CH - 8620 Wetzikon
>
> *Switzerland, Europe Zurich, GMT+1*
> *Twitter:  @atsticks*
> *Blogs: **http://javaremarkables.blogspot.ch/
> <http://javaremarkables.blogspot.ch/>*
>
> *Google: atsticksMobile  +41-76 344 62 79 <%2B41-76%20344%2062%2079>*

Re: PropertyFiltering

[Anatole Tresch <at...@gmail.com>]

Hi Romain

I try to explain again and then you can explain us what should not work ;)

1) all property sources are called to evaluate a single value, or all
properties. The ConfigurationContext provides the ordered list or property
sources. Ordering is based on ordinals. So this is the same as with
Deltaspike and will work. PropertySourceProvider allow to dynamically
register propertysources by default.
2) the value evaluated finally may be go through the filter, e.g. for
resolving placeholders (will be an extension module). Or whatever is
appropriate.

All other kind of filtering will happen on user level. E.g. categories,
metadata and similar stuff can be added as separate modules on top of that.

Looking forward to your thaughts ;)

Anatole



2015-01-02 9:48 GMT+01:00 Romain Manni-Bucau <rm...@gmail.com>:

> Hi Anatole,
>
> just cause I'm not sure I got it: "end of the chain" = ?
>
> If that's after property sources as mentionned earlier it sadly
> doesn't work and you still need them at runtime.
>
>
> Romain Manni-Bucau
> @rmannibucau
> http://www.tomitribe.com
> http://rmannibucau.wordpress.com
> https://github.com/rmannibucau
>
>
> 2015-01-02 8:13 GMT+01:00 Anatole Tresch <at...@gmail.com>:
> > Hi Romain
> >
> > Nobody. (Including me) wants a tree solution ;) be it, we discussed a
> few things yesterday in the hangout. The only things that change, will be
> that you get filters at the end of the chain.
> >
> > Additionally I will add extension modules for metadata / categories
> later. But this has no effect on the core api....
> >
> > Anatole
> >
> > -
> > Anatole Tresch
> > Glärnischweg 10
> > 8620 Wetzikon
> > Tel +41 (43) 317 05 30
> > -
> > Send from Mobile
> >
> >> Am 01.01.2015 um 20:42 schrieb Romain Manni-Bucau <
> rmannibucau@gmail.com>:
> >>
> >> I wouldnt start on a tree solution and i dont agree. Aggregation is fine
> >> and the most performant solution, only issue is not scannable sources
> and
> >> we can say we dont support them IMO.
> >> Le 1 janv. 2015 20:22, "Anatole Tresch" <at...@gmail.com> a écrit :
> >>
> >>> Hi Romain
> >>>
> >>> Not necessarily, but you simply do mot want to see everything
> everywhere.
> >>> You config needs on domain level are different than on app level. You
> >>> simply do not want to see sll the low level grap. Since we do not have
> >>> named configurations anymore, we need true filtering at the levels spi
> >>> implementors want it to have.
> >>> Furthernore: evaluating a value with the current override mechanism is
> too
> >>> costly becausr you are NOT using your chain in an efficient way: when
> you
> >>> have 2 sources a and b. And b defines a value, you always have to
> evaluate
> >>> b, though the fact that b overrides a, so is not relevant for that
> case.
> >>> Similarly evaluating a non scannable value from a is easy. Just b must
> now,
> >>> which values are not cacheable. That coulb be done by some meta data,
> ie
> >>> with another entry compliant with some metadata scheme.
> >>>
> >>> So you get everything: efficiency , filtering, caching,
> contextuality...
> >>> The only thing that this works is to provide access to the current
> >>> evaluation chain eg some evaluation context:
> >>> Something like:
> >>>
> >>> Interface evslctx{
> >>>
> >>> Map<string,string> getCurrentProperties();
> >>> String getCurrentValue();
> >>> Boolean contextHasChanged();
> >>> }
> >>>
> >>> Hereby context has changed can be used to flag a change in the
> preceedong
> >>> sources , so the current source can react on the change, eg by
> reevaluating
> >>> getCurrentProperties();
> >>>
> >>> Within the ConfigContext we then can access the topmost propertysource
> >>> available (therefore available and empty are not the same!). On first
> >>> sccess it will go down the gull chsin, whereas later this might pnly be
> >>> necessary when needed...
> >>>
> >>> Anatole
> >>>
> >>> Romain Manni-Bucau <rm...@gmail.com> schrieb am Do., 1. Jan.
> 2015 um
> >>> 19:46:
> >>>
> >>>> Do you mean we loose the aggregation? Sounds like loosing the
> >>>> configuration management/overview then, are you sure it is good?
> >>>>
> >>>> About filtering I would just care about modifying filters and ignore
> >>>> logging ones = tamaya shouldnt log any value (can log its source if we
> >>>> need) IMO. This logging layer is a very particular need and each app
> >>>> know if a value can be logged or not so adding an abstraction which is
> >>>> not needed in practise seems like overcomplicated to me.
> >>>>
> >>>>
> >>>> Romain Manni-Bucau
> >>>> @rmannibucau
> >>>> http://www.tomitribe.com
> >>>> http://rmannibucau.wordpress.com
> >>>> https://github.com/rmannibucau
> >>>>
> >>>>
> >>>> 2015-01-01 14:18 GMT+01:00 Anatole Tresch <at...@gmail.com>:
> >>>>> See inline...
> >>>>>
> >>>>> 2015-01-01 12:57 GMT+01:00 Mark Struberg <st...@yahoo.de>:
> >>>>>
> >>>>>> If lazy or eager is a matter of each specific Filter. There is no
> >>> common
> >>>>>> answer to that.
> >>>>>>
> >>>>>> The only reason why we have a problem with it is that the the
> proposed
> >>>>>> interface handles the whole map.
> >>>>>
> >>>>> If we would instead just apply the filtering chain on single
> attributes
> >>>>>> (while they get queried) then all those problems would vanish.
> >>>>>>
> >>>>>
> >>>>> Given what you said above. its an implementation aspect
> >>>>> of the filter
> >>>>> , if a filter passes on a key accessed to the chain for reevaluation,
> >>>>> whereas caching others we have exactly that selective filtering
> >>>>> ... so having control where a filter is placed within the chain of
> >>>>> property sources for me is the crucial aspect. I also think for the
> >>> sake
> >>>> of
> >>>>> simplicity filters should have the same granularity as property
> >>> sources,
> >>>>> e.g.
> >>>>>
> >>>>> envprops -> sysprops -> domain-cfg -> domainFilter -> app-cfg ->
> >>>>> securityFilter
> >>>>>
> >>>>> Thats the reason why I started to think to model property source and
> >>>>> filters very similarly, or even model the property source as property
> >>>> chain
> >>>>> member mapping the current input properties to the output properties
> to
> >>>>> pass to the next member (in that case PropertySource may also
> undergo a
> >>>>> renaming).
> >>>>> Alternatively we could model the property chain also the other way
> >>> round.
> >>>>> Instead of aggregating all up until we end up with a result, we may
> >>>> simply
> >>>>> ask the highest available (therefore you need an availability flag)
> >>>>> PropertySource for the current configuration. This implementation
> then
> >>>> can
> >>>>> deliberately decide, if it can provide a key, or it has it to
> delegate
> >>> to
> >>>>> its parent PropertySource:
> >>>>>
> >>>>> Map<String,String> getProperties(PropertySource
> parentPropertySource);
> >>>>> Optional<String> get(String key,
> >>>>> PropertySource parentPropertySource);
> >>>>>
> >>>>> This would solve all: implementation, filtering, caching and still
> the
> >>>>> ConfigurationContext
> >>>>> around has control over the current chain (he passes the parent
> >>> source)..
> >>>>>
> >>>>> There is btw also a difference between filtering to change values
> (e.g.
> >>>>>> decrypt ciphered passwords) and filtering for the purpose of
> logging.
> >>>>>>
> >>>>> Agreed. I think should try to list/discuss the use cases in more
> >>> detail,
> >>>>> when and how configuration should be logged. In most cases I would
> >>> assume
> >>>>> that we can filter by just adding a top level filter, before passing
> >>> the
> >>>>> config to the logger...
> >>>>>
> >>>>>
> >>>>>
> >>>>>>
> >>>>>> LieGrue,
> >>>>>> strub
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>> On Thursday, 1 January 2015, 11:25, Romain Manni-Bucau <
> >>>>>> rmannibucau@gmail.com> wrote:
> >>>>>>>> I think we need both but eager initialization should be the
> >>> default
> >>>>>>> (isScannable = true). All other usages (isScannable = false) should
> >>> be
> >>>>>>> lazy and used with caution (main case I have in mind is JNDI and I
> >>>>>>> wouldnt rely on it really today).
> >>>>>>>
> >>>>>>> What I did in my common impl of conf was to build a Map<String,
> >>>>>>> String> then "sources" were able to trigger a refresh event making
> >>>>>>> the
> >>>>>>> configuration rebuilding the Map then do a diff and switch if
> >>>>>>> different. This implies only eager eval and makes conf management
> >>> very
> >>>>>>> easy but still allowing updates. Only case it doesnt handle is lazy
> >>>>>>> eval (JNDI) but this was intended cause harder to
> >>>>>>> manage/control/monitor - view would be partial and then wrong if
> >>>>>>> allowed.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> Romain Manni-Bucau
> >>>>>>> @rmannibucau
> >>>>>>> http://www.tomitribe.com
> >>>>>>> http://rmannibucau.wordpress.com
> >>>>>>> https://github.com/rmannibucau
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> 2015-01-01 11:18 GMT+01:00 Anatole Tresch <at...@gmail.com>:
> >>>>>>>> I would say lazyness for evaluation is a must (due to possible
> >>>>>> environment
> >>>>>>>> dependencies properties returned can be different!), caching of
> >>>>>> aggregated
> >>>>>>>> configuration on some level comes on top. Or did I get something
> >>>>>> wrong...?
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> 2015-01-01 11:11 GMT+01:00 Romain Manni-Bucau
> >>>>>>> <rm...@gmail.com>:
> >>>>>>>>
> >>>>>>>>> wouldnt it make it harder to implement - ie not user friendly but
> >>>> impl
> >>>>>>>>> friendly only?
> >>>>>>>>>
> >>>>>>>>> + not sure lazyness is useful we need an aggregated config anyway
> >>>> (=
> >>>>>>> eager)
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> Romain Manni-Bucau
> >>>>>>>>> @rmannibucau
> >>>>>>>>> http://www.tomitribe.com
> >>>>>>>>> http://rmannibucau.wordpress.com
> >>>>>>>>> https://github.com/rmannibucau
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> 2015-01-01 11:08 GMT+01:00 Anatole Tresch <an...@apache.org>:
> >>>>>>>>>> Hi all
> >>>>>>>>>>
> >>>>>>>>>> another alternative that comes to my mind, would be to model
> >>> the
> >>>>>>>>> PropertySource
> >>>>>>>>>> more as a Stream<Map.Entry<String,String>> than a
> >>>>>>> provider:
> >>>>>>>>>>
> >>>>>>>>>>   /**
> >>>>>>>>>>     * <p>Evaluate the current properties as Map, based on
> >>>>>>> the current
> >>>>>>>>>> defined properties. The resulting Map may not        * return
> >>> all
> >>>>>>> items
> >>>>>>>>>> accessible, when the PropertySource is filtering values for the
> >>>>>>> context,
> >>>>>>>>>> where the
> >>>>>>>>>>     * PropertySource is available.</p>
> >>>>>>>>>>     * <p>PropertySources that are not filtering data,
> >>>>>>> should either add
> >>>>>>>>> or
> >>>>>>>>>> override entries, but not remove
> >>>>>>>>>>     * any entries provided by the input map.
> >>>>>>>>>>     *
> >>>>>>>>>>     * @return the a corresponding map, never null.
> >>>>>>>>>>     * //X TODO or should we just do getPropertyKeys()? Think
> >>>>>>> about
> >>>>>>>>>> security (key) vs easier merging (full map)?
> >>>>>>>>>>     */
> >>>>>>>>>>    Stream<Map.Entry<String,String>> getProperties(
> >>>>>>>>>> Stream<Map.Entry<String,String>>
> >>>>>>> currentEntriesStream);
> >>>>>>>>>>
> >>>>>>>>>> This allows us to export all collecting and filtering
> >>> activities
> >>>>>>> into the
> >>>>>>>>>> Stream API, and also adding entries is simple:
> >>>>>>>>>>
> >>>>>>>>>> Stream stream = Stream.concat(stream1, stream2);
> >>>>>>>>>>
> >>>>>>>>>> Our code gets bare simple and we profit from all the lazy
> >>>>>>> evaluation
> >>>>>>>>>> features....
> >>>>>>>>>>
> >>>>>>>>>> WDYT?
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> 2015-01-01 10:44 GMT+01:00 Anatole Tresch
> >>>>>>> <an...@apache.org>:
> >>>>>>>>>>
> >>>>>>>>>>> Hi all
> >>>>>>>>>>>
> >>>>>>>>>>> I would like to make a *SPI proposal* for PropertyFilter (on
> >>>>>>> API side we
> >>>>>>>>>>> have UnaryOperator<Configuration>). Hereby filters have
> >>>>>>> ordinals and
> >>>>>>>>>>> undergo the same ordering mechanism as PropertySources.
> >>>>>>> Filters get
> >>>>>>>>>>> similarly added to chain of evaluation as PropertySources
> >>>>>>>>> (implementations
> >>>>>>>>>>> must add @Priority -> do we need a programmatic
> >>>>>>> getOrdinal() to have a
> >>>>>>>>> more
> >>>>>>>>>>> concise usae here?), with one exceptional rule:
> >>>>>>>>>>> If a filter accidentally has the same ordinal as some
> >>>>>>> PropertySources in
> >>>>>>>>>>> the chain, filters are always executed AFTER PropertySources
> >>>>>>>>>>>
> >>>>>>>>>>> So here is the interface:.
> >>>>>>>>>>>
> >>>>>>>>>>> /**
> >>>>>>>>>>> * <p>Interface for filtering the current map of
> >>>>>>> properties during the
> >>>>>>>>>>> evaluation of the chain of PropertySources.
> >>>>>>>>>>> * Filters can be registered using the {@link
> >>>>>>>>>>> org.apache.tamaya.spi.ServiceContext} or added
> >>>>>>> programmatically
> >>>>>>>>>>> * by calling {@link
> >>>>>>>>>>>
> >>>>>>> org.apache.tamaya.spi.ConfigurationContext#addPropertyFilter(int,
> >>>>>>>>>>> PropertyFilter)}. The ordinal
> >>>>>>>>>>> * hereby is defined by the corresponding {@code
> >>>>>>> @Priority}
> >>>>>>>>> annotation.</p>
> >>>>>>>>>>> * <p>Filters </p>
> >>>>>>>>>>> */
> >>>>>>>>>>> public interface PropertyFilter<T>{
> >>>>>>>>>>>
> >>>>>>>>>>>    /**
> >>>>>>>>>>>     * Method to check, if a filter is available. Filters that
> >>>>>>> are
> >>>>>>>>>>> currently not available are not considered, when
> >>>>>>>>>>>     * evaluating the current chain of {@link
> >>>>>>>>>>> org.apache.tamaya.spi.PropertySource} instances.
> >>>>>>>>>>>     * @return true, if a filter is available.
> >>>>>>>>>>>     */
> >>>>>>>>>>>    public boolean isAvailable();
> >>>>>>>>>>>
> >>>>>>>>>>>    /**
> >>>>>>>>>>>     * Maps the current properties map to the filtered version
> >>>>>>> of the
> >>>>>>>>> map.
> >>>>>>>>>>> The resulting map will be used as the input
> >>>>>>>>>>>     * for all subsequent handling of PropertySources in the
> >>>>>>>>>>> PropertySource chain.
> >>>>>>>>>>>     * @param currentMap the current property map, not null
> >>>>>>>>>>>     * @return the filtered map, never null.
> >>>>>>>>>>>     */
> >>>>>>>>>>>    Map<String,String> map(Map<String,String>
> >>>>>>> currentMap);
> >>>>>>>>>>>
> >>>>>>>>>>> }
> >>>>>>>>>>>
> >>>>>>>>>>> WDYT?
> >>>>>>>>>>>
> >>>>>>>>>>> Anatole
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> 2014-12-29 17:59 GMT+01:00 Anatole Tresch
> >>>>>>> <an...@apache.org>:
> >>>>>>>>>>>
> >>>>>>>>>>>> Whereas overrides are simple with this mechanism. I am
> >>>>>>> asking how we
> >>>>>>>>> want
> >>>>>>>>>>>> to do other aggregation types, or filtering of values.
> >>>>>>>>>>>>
> >>>>>>>>>>>> *On the API side* filtering is always simple, since we
> >>>>>>> have the
> >>>>>>>>>>>> functional extension points
> >>>>>>> (UnaryOperator<Configuration>), where we
> >>>>>>>>> can
> >>>>>>>>>>>> easily filter the whole configuration, and the adapters
> >>>>>>> for filtering
> >>>>>>>>>>>> single entries on access.
> >>>>>>>>>>>>
> >>>>>>>>>>>> But let's assume the following use case:
> >>>>>>>>>>>>
> >>>>>>>>>>>> *We have a system low level configuration S, which also
> >>>>>>> contains a few
> >>>>>>>>>>>> entries, I do not want to see on application config A.
> >>>>>>> Given our
> >>>>>>>>> priority
> >>>>>>>>>>>> based assembly this is not possible, since I only can
> >>>>>>> override the
> >>>>>>>>> values
> >>>>>>>>>>>> from my parent sources, but never remove them.*
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thinking on this the following ideas come to my mind:
> >>>>>>>>>>>>
> >>>>>>>>>>>> 1) *add PropertySource filters to my chain. I could use
> >>>>>>> the same
> >>>>>>>>>>>> priorities. The bad thing is that it is not clear if the
> >>>>>>> filter should
> >>>>>>>>> be
> >>>>>>>>>>>> active also, when no source with a similar priority is
> >>>>>>> active and if
> >>>>>>>>> so if
> >>>>>>>>>>>> it should be before or after, so this will not work.*
> >>>>>>>>>>>> *2) I could define a PropertySource to be filtering, so it
> >>>>>>> is able not
> >>>>>>>>>>>> only to override, but also to remove entries. This
> >>>>>>> "filter" can be
> >>>>>>>>>>>> activated or deactivated depending on the current context.
> >>>>>>> Given that
> >>>>>>>>> (if
> >>>>>>>>>>>> we have set a correct priority value), all works fine.*
> >>>>>>>>>>>> *3) I could define a separate filter layer which is
> >>>>>>> allowed to filter
> >>>>>>>>> the
> >>>>>>>>>>>> entries, before they are published (after all overrides
> >>>>>>> were done).
> >>>>>>>>> This
> >>>>>>>>>>>> would probably be transparently be hooked into the
> >>>>>>> "Configuration
> >>>>>>>>> Layer",
> >>>>>>>>>>>> which takes the current property sources and creates the
> >>>>>>> final map of
> >>>>>>>>>>>> properties. Then on the final Map the filter service is
> >>>>>>> operating
> >>>>>>>>> before
> >>>>>>>>>>>> any data is handed out to the callers.*
> >>>>>>>>>>>>
> >>>>>>>>>>>> I think (3) is the best solution here. Nevertheless we
> >>>>>>> should think on
> >>>>>>>>>>>> how this can be done the best way and how the SPI here
> >>>>>>> would look like.
> >>>>>>>>>>>> Also I think, we might have several filters active, e.g.
> >>>>>>> one for
> >>>>>>>>> logging,
> >>>>>>>>>>>> one for admins, one which considers the current user's
> >>>>>>> role (aka
> >>>>>>>>> views), ...
> >>>>>>>>>>>>
> >>>>>>>>>>>> So ideas are welcome ;)
> >>>>>>>>>>>>
> >>>>>>>>>>>> Best,
> >>>>>>>>>>>> Anatole
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> *Anatole Tresch*
> >>>>>>>> Java Engineer & Architect, JSR Spec Lead
> >>>>>>>> Glärnischweg 10
> >>>>>>>> CH - 8620 Wetzikon
> >>>>>>>>
> >>>>>>>> *Switzerland, Europe Zurich, GMT+1*
> >>>>>>>> *Twitter:  @atsticks*
> >>>>>>>> *Blogs: **http://javaremarkables.blogspot.ch/
> >>>>>>>> <http://javaremarkables.blogspot.ch/>*
> >>>>>>>>
> >>>>>>>> *Google: atsticksMobile  +41-76 344 62 79*
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> *Anatole Tresch*
> >>>>> Java Engineer & Architect, JSR Spec Lead
> >>>>> Glärnischweg 10
> >>>>> CH - 8620 Wetzikon
> >>>>>
> >>>>> *Switzerland, Europe Zurich, GMT+1*
> >>>>> *Twitter:  @atsticks*
> >>>>> *Blogs: **http://javaremarkables.blogspot.ch/
> >>>>> <http://javaremarkables.blogspot.ch/>*
> >>>>>
> >>>>> *Google: atsticksMobile  +41-76 344 62 79*
> >>>>
> >>>
>



-- 
*Anatole Tresch*
Java Engineer & Architect, JSR Spec Lead
Glärnischweg 10
CH - 8620 Wetzikon

*Switzerland, Europe Zurich, GMT+1*
*Twitter:  @atsticks*
*Blogs: **http://javaremarkables.blogspot.ch/
<http://javaremarkables.blogspot.ch/>*

*Google: atsticksMobile  +41-76 344 62 79 <%2B41-76%20344%2062%2079>*

Re: PropertyFiltering

[Romain Manni-Bucau <rm...@gmail.com>]

Hi Anatole,

just cause I'm not sure I got it: "end of the chain" = ?

If that's after property sources as mentionned earlier it sadly
doesn't work and you still need them at runtime.


Romain Manni-Bucau
@rmannibucau
http://www.tomitribe.com
http://rmannibucau.wordpress.com
https://github.com/rmannibucau


2015-01-02 8:13 GMT+01:00 Anatole Tresch <at...@gmail.com>:
> Hi Romain
>
> Nobody. (Including me) wants a tree solution ;) be it, we discussed a few things yesterday in the hangout. The only things that change, will be that you get filters at the end of the chain.
>
> Additionally I will add extension modules for metadata / categories later. But this has no effect on the core api....
>
> Anatole
>
> -
> Anatole Tresch
> Glärnischweg 10
> 8620 Wetzikon
> Tel +41 (43) 317 05 30
> -
> Send from Mobile
>
>> Am 01.01.2015 um 20:42 schrieb Romain Manni-Bucau <rm...@gmail.com>:
>>
>> I wouldnt start on a tree solution and i dont agree. Aggregation is fine
>> and the most performant solution, only issue is not scannable sources and
>> we can say we dont support them IMO.
>> Le 1 janv. 2015 20:22, "Anatole Tresch" <at...@gmail.com> a écrit :
>>
>>> Hi Romain
>>>
>>> Not necessarily, but you simply do mot want to see everything everywhere.
>>> You config needs on domain level are different than on app level. You
>>> simply do not want to see sll the low level grap. Since we do not have
>>> named configurations anymore, we need true filtering at the levels spi
>>> implementors want it to have.
>>> Furthernore: evaluating a value with the current override mechanism is too
>>> costly becausr you are NOT using your chain in an efficient way: when you
>>> have 2 sources a and b. And b defines a value, you always have to evaluate
>>> b, though the fact that b overrides a, so is not relevant for that case.
>>> Similarly evaluating a non scannable value from a is easy. Just b must now,
>>> which values are not cacheable. That coulb be done by some meta data, ie
>>> with another entry compliant with some metadata scheme.
>>>
>>> So you get everything: efficiency , filtering, caching, contextuality...
>>> The only thing that this works is to provide access to the current
>>> evaluation chain eg some evaluation context:
>>> Something like:
>>>
>>> Interface evslctx{
>>>
>>> Map<string,string> getCurrentProperties();
>>> String getCurrentValue();
>>> Boolean contextHasChanged();
>>> }
>>>
>>> Hereby context has changed can be used to flag a change in the preceedong
>>> sources , so the current source can react on the change, eg by reevaluating
>>> getCurrentProperties();
>>>
>>> Within the ConfigContext we then can access the topmost propertysource
>>> available (therefore available and empty are not the same!). On first
>>> sccess it will go down the gull chsin, whereas later this might pnly be
>>> necessary when needed...
>>>
>>> Anatole
>>>
>>> Romain Manni-Bucau <rm...@gmail.com> schrieb am Do., 1. Jan. 2015 um
>>> 19:46:
>>>
>>>> Do you mean we loose the aggregation? Sounds like loosing the
>>>> configuration management/overview then, are you sure it is good?
>>>>
>>>> About filtering I would just care about modifying filters and ignore
>>>> logging ones = tamaya shouldnt log any value (can log its source if we
>>>> need) IMO. This logging layer is a very particular need and each app
>>>> know if a value can be logged or not so adding an abstraction which is
>>>> not needed in practise seems like overcomplicated to me.
>>>>
>>>>
>>>> Romain Manni-Bucau
>>>> @rmannibucau
>>>> http://www.tomitribe.com
>>>> http://rmannibucau.wordpress.com
>>>> https://github.com/rmannibucau
>>>>
>>>>
>>>> 2015-01-01 14:18 GMT+01:00 Anatole Tresch <at...@gmail.com>:
>>>>> See inline...
>>>>>
>>>>> 2015-01-01 12:57 GMT+01:00 Mark Struberg <st...@yahoo.de>:
>>>>>
>>>>>> If lazy or eager is a matter of each specific Filter. There is no
>>> common
>>>>>> answer to that.
>>>>>>
>>>>>> The only reason why we have a problem with it is that the the proposed
>>>>>> interface handles the whole map.
>>>>>
>>>>> If we would instead just apply the filtering chain on single attributes
>>>>>> (while they get queried) then all those problems would vanish.
>>>>>>
>>>>>
>>>>> Given what you said above. its an implementation aspect
>>>>> of the filter
>>>>> , if a filter passes on a key accessed to the chain for reevaluation,
>>>>> whereas caching others we have exactly that selective filtering
>>>>> ... so having control where a filter is placed within the chain of
>>>>> property sources for me is the crucial aspect. I also think for the
>>> sake
>>>> of
>>>>> simplicity filters should have the same granularity as property
>>> sources,
>>>>> e.g.
>>>>>
>>>>> envprops -> sysprops -> domain-cfg -> domainFilter -> app-cfg ->
>>>>> securityFilter
>>>>>
>>>>> Thats the reason why I started to think to model property source and
>>>>> filters very similarly, or even model the property source as property
>>>> chain
>>>>> member mapping the current input properties to the output properties to
>>>>> pass to the next member (in that case PropertySource may also undergo a
>>>>> renaming).
>>>>> Alternatively we could model the property chain also the other way
>>> round.
>>>>> Instead of aggregating all up until we end up with a result, we may
>>>> simply
>>>>> ask the highest available (therefore you need an availability flag)
>>>>> PropertySource for the current configuration. This implementation then
>>>> can
>>>>> deliberately decide, if it can provide a key, or it has it to delegate
>>> to
>>>>> its parent PropertySource:
>>>>>
>>>>> Map<String,String> getProperties(PropertySource parentPropertySource);
>>>>> Optional<String> get(String key,
>>>>> PropertySource parentPropertySource);
>>>>>
>>>>> This would solve all: implementation, filtering, caching and still the
>>>>> ConfigurationContext
>>>>> around has control over the current chain (he passes the parent
>>> source)..
>>>>>
>>>>> There is btw also a difference between filtering to change values (e.g.
>>>>>> decrypt ciphered passwords) and filtering for the purpose of logging.
>>>>>>
>>>>> Agreed. I think should try to list/discuss the use cases in more
>>> detail,
>>>>> when and how configuration should be logged. In most cases I would
>>> assume
>>>>> that we can filter by just adding a top level filter, before passing
>>> the
>>>>> config to the logger...
>>>>>
>>>>>
>>>>>
>>>>>>
>>>>>> LieGrue,
>>>>>> strub
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> On Thursday, 1 January 2015, 11:25, Romain Manni-Bucau <
>>>>>> rmannibucau@gmail.com> wrote:
>>>>>>>> I think we need both but eager initialization should be the
>>> default
>>>>>>> (isScannable = true). All other usages (isScannable = false) should
>>> be
>>>>>>> lazy and used with caution (main case I have in mind is JNDI and I
>>>>>>> wouldnt rely on it really today).
>>>>>>>
>>>>>>> What I did in my common impl of conf was to build a Map<String,
>>>>>>> String> then "sources" were able to trigger a refresh event making
>>>>>>> the
>>>>>>> configuration rebuilding the Map then do a diff and switch if
>>>>>>> different. This implies only eager eval and makes conf management
>>> very
>>>>>>> easy but still allowing updates. Only case it doesnt handle is lazy
>>>>>>> eval (JNDI) but this was intended cause harder to
>>>>>>> manage/control/monitor - view would be partial and then wrong if
>>>>>>> allowed.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Romain Manni-Bucau
>>>>>>> @rmannibucau
>>>>>>> http://www.tomitribe.com
>>>>>>> http://rmannibucau.wordpress.com
>>>>>>> https://github.com/rmannibucau
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 2015-01-01 11:18 GMT+01:00 Anatole Tresch <at...@gmail.com>:
>>>>>>>> I would say lazyness for evaluation is a must (due to possible
>>>>>> environment
>>>>>>>> dependencies properties returned can be different!), caching of
>>>>>> aggregated
>>>>>>>> configuration on some level comes on top. Or did I get something
>>>>>> wrong...?
>>>>>>>>
>>>>>>>>
>>>>>>>> 2015-01-01 11:11 GMT+01:00 Romain Manni-Bucau
>>>>>>> <rm...@gmail.com>:
>>>>>>>>
>>>>>>>>> wouldnt it make it harder to implement - ie not user friendly but
>>>> impl
>>>>>>>>> friendly only?
>>>>>>>>>
>>>>>>>>> + not sure lazyness is useful we need an aggregated config anyway
>>>> (=
>>>>>>> eager)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Romain Manni-Bucau
>>>>>>>>> @rmannibucau
>>>>>>>>> http://www.tomitribe.com
>>>>>>>>> http://rmannibucau.wordpress.com
>>>>>>>>> https://github.com/rmannibucau
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> 2015-01-01 11:08 GMT+01:00 Anatole Tresch <an...@apache.org>:
>>>>>>>>>> Hi all
>>>>>>>>>>
>>>>>>>>>> another alternative that comes to my mind, would be to model
>>> the
>>>>>>>>> PropertySource
>>>>>>>>>> more as a Stream<Map.Entry<String,String>> than a
>>>>>>> provider:
>>>>>>>>>>
>>>>>>>>>>   /**
>>>>>>>>>>     * <p>Evaluate the current properties as Map, based on
>>>>>>> the current
>>>>>>>>>> defined properties. The resulting Map may not        * return
>>> all
>>>>>>> items
>>>>>>>>>> accessible, when the PropertySource is filtering values for the
>>>>>>> context,
>>>>>>>>>> where the
>>>>>>>>>>     * PropertySource is available.</p>
>>>>>>>>>>     * <p>PropertySources that are not filtering data,
>>>>>>> should either add
>>>>>>>>> or
>>>>>>>>>> override entries, but not remove
>>>>>>>>>>     * any entries provided by the input map.
>>>>>>>>>>     *
>>>>>>>>>>     * @return the a corresponding map, never null.
>>>>>>>>>>     * //X TODO or should we just do getPropertyKeys()? Think
>>>>>>> about
>>>>>>>>>> security (key) vs easier merging (full map)?
>>>>>>>>>>     */
>>>>>>>>>>    Stream<Map.Entry<String,String>> getProperties(
>>>>>>>>>> Stream<Map.Entry<String,String>>
>>>>>>> currentEntriesStream);
>>>>>>>>>>
>>>>>>>>>> This allows us to export all collecting and filtering
>>> activities
>>>>>>> into the
>>>>>>>>>> Stream API, and also adding entries is simple:
>>>>>>>>>>
>>>>>>>>>> Stream stream = Stream.concat(stream1, stream2);
>>>>>>>>>>
>>>>>>>>>> Our code gets bare simple and we profit from all the lazy
>>>>>>> evaluation
>>>>>>>>>> features....
>>>>>>>>>>
>>>>>>>>>> WDYT?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> 2015-01-01 10:44 GMT+01:00 Anatole Tresch
>>>>>>> <an...@apache.org>:
>>>>>>>>>>
>>>>>>>>>>> Hi all
>>>>>>>>>>>
>>>>>>>>>>> I would like to make a *SPI proposal* for PropertyFilter (on
>>>>>>> API side we
>>>>>>>>>>> have UnaryOperator<Configuration>). Hereby filters have
>>>>>>> ordinals and
>>>>>>>>>>> undergo the same ordering mechanism as PropertySources.
>>>>>>> Filters get
>>>>>>>>>>> similarly added to chain of evaluation as PropertySources
>>>>>>>>> (implementations
>>>>>>>>>>> must add @Priority -> do we need a programmatic
>>>>>>> getOrdinal() to have a
>>>>>>>>> more
>>>>>>>>>>> concise usae here?), with one exceptional rule:
>>>>>>>>>>> If a filter accidentally has the same ordinal as some
>>>>>>> PropertySources in
>>>>>>>>>>> the chain, filters are always executed AFTER PropertySources
>>>>>>>>>>>
>>>>>>>>>>> So here is the interface:.
>>>>>>>>>>>
>>>>>>>>>>> /**
>>>>>>>>>>> * <p>Interface for filtering the current map of
>>>>>>> properties during the
>>>>>>>>>>> evaluation of the chain of PropertySources.
>>>>>>>>>>> * Filters can be registered using the {@link
>>>>>>>>>>> org.apache.tamaya.spi.ServiceContext} or added
>>>>>>> programmatically
>>>>>>>>>>> * by calling {@link
>>>>>>>>>>>
>>>>>>> org.apache.tamaya.spi.ConfigurationContext#addPropertyFilter(int,
>>>>>>>>>>> PropertyFilter)}. The ordinal
>>>>>>>>>>> * hereby is defined by the corresponding {@code
>>>>>>> @Priority}
>>>>>>>>> annotation.</p>
>>>>>>>>>>> * <p>Filters </p>
>>>>>>>>>>> */
>>>>>>>>>>> public interface PropertyFilter<T>{
>>>>>>>>>>>
>>>>>>>>>>>    /**
>>>>>>>>>>>     * Method to check, if a filter is available. Filters that
>>>>>>> are
>>>>>>>>>>> currently not available are not considered, when
>>>>>>>>>>>     * evaluating the current chain of {@link
>>>>>>>>>>> org.apache.tamaya.spi.PropertySource} instances.
>>>>>>>>>>>     * @return true, if a filter is available.
>>>>>>>>>>>     */
>>>>>>>>>>>    public boolean isAvailable();
>>>>>>>>>>>
>>>>>>>>>>>    /**
>>>>>>>>>>>     * Maps the current properties map to the filtered version
>>>>>>> of the
>>>>>>>>> map.
>>>>>>>>>>> The resulting map will be used as the input
>>>>>>>>>>>     * for all subsequent handling of PropertySources in the
>>>>>>>>>>> PropertySource chain.
>>>>>>>>>>>     * @param currentMap the current property map, not null
>>>>>>>>>>>     * @return the filtered map, never null.
>>>>>>>>>>>     */
>>>>>>>>>>>    Map<String,String> map(Map<String,String>
>>>>>>> currentMap);
>>>>>>>>>>>
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>> WDYT?
>>>>>>>>>>>
>>>>>>>>>>> Anatole
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> 2014-12-29 17:59 GMT+01:00 Anatole Tresch
>>>>>>> <an...@apache.org>:
>>>>>>>>>>>
>>>>>>>>>>>> Whereas overrides are simple with this mechanism. I am
>>>>>>> asking how we
>>>>>>>>> want
>>>>>>>>>>>> to do other aggregation types, or filtering of values.
>>>>>>>>>>>>
>>>>>>>>>>>> *On the API side* filtering is always simple, since we
>>>>>>> have the
>>>>>>>>>>>> functional extension points
>>>>>>> (UnaryOperator<Configuration>), where we
>>>>>>>>> can
>>>>>>>>>>>> easily filter the whole configuration, and the adapters
>>>>>>> for filtering
>>>>>>>>>>>> single entries on access.
>>>>>>>>>>>>
>>>>>>>>>>>> But let's assume the following use case:
>>>>>>>>>>>>
>>>>>>>>>>>> *We have a system low level configuration S, which also
>>>>>>> contains a few
>>>>>>>>>>>> entries, I do not want to see on application config A.
>>>>>>> Given our
>>>>>>>>> priority
>>>>>>>>>>>> based assembly this is not possible, since I only can
>>>>>>> override the
>>>>>>>>> values
>>>>>>>>>>>> from my parent sources, but never remove them.*
>>>>>>>>>>>>
>>>>>>>>>>>> Thinking on this the following ideas come to my mind:
>>>>>>>>>>>>
>>>>>>>>>>>> 1) *add PropertySource filters to my chain. I could use
>>>>>>> the same
>>>>>>>>>>>> priorities. The bad thing is that it is not clear if the
>>>>>>> filter should
>>>>>>>>> be
>>>>>>>>>>>> active also, when no source with a similar priority is
>>>>>>> active and if
>>>>>>>>> so if
>>>>>>>>>>>> it should be before or after, so this will not work.*
>>>>>>>>>>>> *2) I could define a PropertySource to be filtering, so it
>>>>>>> is able not
>>>>>>>>>>>> only to override, but also to remove entries. This
>>>>>>> "filter" can be
>>>>>>>>>>>> activated or deactivated depending on the current context.
>>>>>>> Given that
>>>>>>>>> (if
>>>>>>>>>>>> we have set a correct priority value), all works fine.*
>>>>>>>>>>>> *3) I could define a separate filter layer which is
>>>>>>> allowed to filter
>>>>>>>>> the
>>>>>>>>>>>> entries, before they are published (after all overrides
>>>>>>> were done).
>>>>>>>>> This
>>>>>>>>>>>> would probably be transparently be hooked into the
>>>>>>> "Configuration
>>>>>>>>> Layer",
>>>>>>>>>>>> which takes the current property sources and creates the
>>>>>>> final map of
>>>>>>>>>>>> properties. Then on the final Map the filter service is
>>>>>>> operating
>>>>>>>>> before
>>>>>>>>>>>> any data is handed out to the callers.*
>>>>>>>>>>>>
>>>>>>>>>>>> I think (3) is the best solution here. Nevertheless we
>>>>>>> should think on
>>>>>>>>>>>> how this can be done the best way and how the SPI here
>>>>>>> would look like.
>>>>>>>>>>>> Also I think, we might have several filters active, e.g.
>>>>>>> one for
>>>>>>>>> logging,
>>>>>>>>>>>> one for admins, one which considers the current user's
>>>>>>> role (aka
>>>>>>>>> views), ...
>>>>>>>>>>>>
>>>>>>>>>>>> So ideas are welcome ;)
>>>>>>>>>>>>
>>>>>>>>>>>> Best,
>>>>>>>>>>>> Anatole
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> *Anatole Tresch*
>>>>>>>> Java Engineer & Architect, JSR Spec Lead
>>>>>>>> Glärnischweg 10
>>>>>>>> CH - 8620 Wetzikon
>>>>>>>>
>>>>>>>> *Switzerland, Europe Zurich, GMT+1*
>>>>>>>> *Twitter:  @atsticks*
>>>>>>>> *Blogs: **http://javaremarkables.blogspot.ch/
>>>>>>>> <http://javaremarkables.blogspot.ch/>*
>>>>>>>>
>>>>>>>> *Google: atsticksMobile  +41-76 344 62 79*
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *Anatole Tresch*
>>>>> Java Engineer & Architect, JSR Spec Lead
>>>>> Glärnischweg 10
>>>>> CH - 8620 Wetzikon
>>>>>
>>>>> *Switzerland, Europe Zurich, GMT+1*
>>>>> *Twitter:  @atsticks*
>>>>> *Blogs: **http://javaremarkables.blogspot.ch/
>>>>> <http://javaremarkables.blogspot.ch/>*
>>>>>
>>>>> *Google: atsticksMobile  +41-76 344 62 79*
>>>>
>>>

Re: PropertyFiltering

[Anatole Tresch <at...@gmail.com>]

Hi Romain

Nobody. (Including me) wants a tree solution ;) be it, we discussed a few things yesterday in the hangout. The only things that change, will be that you get filters at the end of the chain.

Additionally I will add extension modules for metadata / categories later. But this has no effect on the core api....

Anatole

-
Anatole Tresch
Glärnischweg 10
8620 Wetzikon
Tel +41 (43) 317 05 30
-
Send from Mobile

> Am 01.01.2015 um 20:42 schrieb Romain Manni-Bucau <rm...@gmail.com>:
> 
> I wouldnt start on a tree solution and i dont agree. Aggregation is fine
> and the most performant solution, only issue is not scannable sources and
> we can say we dont support them IMO.
> Le 1 janv. 2015 20:22, "Anatole Tresch" <at...@gmail.com> a écrit :
> 
>> Hi Romain
>> 
>> Not necessarily, but you simply do mot want to see everything everywhere.
>> You config needs on domain level are different than on app level. You
>> simply do not want to see sll the low level grap. Since we do not have
>> named configurations anymore, we need true filtering at the levels spi
>> implementors want it to have.
>> Furthernore: evaluating a value with the current override mechanism is too
>> costly becausr you are NOT using your chain in an efficient way: when you
>> have 2 sources a and b. And b defines a value, you always have to evaluate
>> b, though the fact that b overrides a, so is not relevant for that case.
>> Similarly evaluating a non scannable value from a is easy. Just b must now,
>> which values are not cacheable. That coulb be done by some meta data, ie
>> with another entry compliant with some metadata scheme.
>> 
>> So you get everything: efficiency , filtering, caching, contextuality...
>> The only thing that this works is to provide access to the current
>> evaluation chain eg some evaluation context:
>> Something like:
>> 
>> Interface evslctx{
>> 
>> Map<string,string> getCurrentProperties();
>> String getCurrentValue();
>> Boolean contextHasChanged();
>> }
>> 
>> Hereby context has changed can be used to flag a change in the preceedong
>> sources , so the current source can react on the change, eg by reevaluating
>> getCurrentProperties();
>> 
>> Within the ConfigContext we then can access the topmost propertysource
>> available (therefore available and empty are not the same!). On first
>> sccess it will go down the gull chsin, whereas later this might pnly be
>> necessary when needed...
>> 
>> Anatole
>> 
>> Romain Manni-Bucau <rm...@gmail.com> schrieb am Do., 1. Jan. 2015 um
>> 19:46:
>> 
>>> Do you mean we loose the aggregation? Sounds like loosing the
>>> configuration management/overview then, are you sure it is good?
>>> 
>>> About filtering I would just care about modifying filters and ignore
>>> logging ones = tamaya shouldnt log any value (can log its source if we
>>> need) IMO. This logging layer is a very particular need and each app
>>> know if a value can be logged or not so adding an abstraction which is
>>> not needed in practise seems like overcomplicated to me.
>>> 
>>> 
>>> Romain Manni-Bucau
>>> @rmannibucau
>>> http://www.tomitribe.com
>>> http://rmannibucau.wordpress.com
>>> https://github.com/rmannibucau
>>> 
>>> 
>>> 2015-01-01 14:18 GMT+01:00 Anatole Tresch <at...@gmail.com>:
>>>> See inline...
>>>> 
>>>> 2015-01-01 12:57 GMT+01:00 Mark Struberg <st...@yahoo.de>:
>>>> 
>>>>> If lazy or eager is a matter of each specific Filter. There is no
>> common
>>>>> answer to that.
>>>>> 
>>>>> The only reason why we have a problem with it is that the the proposed
>>>>> interface handles the whole map.
>>>> 
>>>> If we would instead just apply the filtering chain on single attributes
>>>>> (while they get queried) then all those problems would vanish.
>>>>> 
>>>> 
>>>> Given what you said above. its an implementation aspect
>>>> of the filter
>>>> , if a filter passes on a key accessed to the chain for reevaluation,
>>>> whereas caching others we have exactly that selective filtering
>>>> ... so having control where a filter is placed within the chain of
>>>> property sources for me is the crucial aspect. I also think for the
>> sake
>>> of
>>>> simplicity filters should have the same granularity as property
>> sources,
>>>> e.g.
>>>> 
>>>> envprops -> sysprops -> domain-cfg -> domainFilter -> app-cfg ->
>>>> securityFilter
>>>> 
>>>> Thats the reason why I started to think to model property source and
>>>> filters very similarly, or even model the property source as property
>>> chain
>>>> member mapping the current input properties to the output properties to
>>>> pass to the next member (in that case PropertySource may also undergo a
>>>> renaming).
>>>> Alternatively we could model the property chain also the other way
>> round.
>>>> Instead of aggregating all up until we end up with a result, we may
>>> simply
>>>> ask the highest available (therefore you need an availability flag)
>>>> PropertySource for the current configuration. This implementation then
>>> can
>>>> deliberately decide, if it can provide a key, or it has it to delegate
>> to
>>>> its parent PropertySource:
>>>> 
>>>> Map<String,String> getProperties(PropertySource parentPropertySource);
>>>> Optional<String> get(String key,
>>>> PropertySource parentPropertySource);
>>>> 
>>>> This would solve all: implementation, filtering, caching and still the
>>>> ConfigurationContext
>>>> around has control over the current chain (he passes the parent
>> source)..
>>>> 
>>>> There is btw also a difference between filtering to change values (e.g.
>>>>> decrypt ciphered passwords) and filtering for the purpose of logging.
>>>>> 
>>>> Agreed. I think should try to list/discuss the use cases in more
>> detail,
>>>> when and how configuration should be logged. In most cases I would
>> assume
>>>> that we can filter by just adding a top level filter, before passing
>> the
>>>> config to the logger...
>>>> 
>>>> 
>>>> 
>>>>> 
>>>>> LieGrue,
>>>>> strub
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>>> On Thursday, 1 January 2015, 11:25, Romain Manni-Bucau <
>>>>> rmannibucau@gmail.com> wrote:
>>>>>>> I think we need both but eager initialization should be the
>> default
>>>>>> (isScannable = true). All other usages (isScannable = false) should
>> be
>>>>>> lazy and used with caution (main case I have in mind is JNDI and I
>>>>>> wouldnt rely on it really today).
>>>>>> 
>>>>>> What I did in my common impl of conf was to build a Map<String,
>>>>>> String> then "sources" were able to trigger a refresh event making
>>>>>> the
>>>>>> configuration rebuilding the Map then do a diff and switch if
>>>>>> different. This implies only eager eval and makes conf management
>> very
>>>>>> easy but still allowing updates. Only case it doesnt handle is lazy
>>>>>> eval (JNDI) but this was intended cause harder to
>>>>>> manage/control/monitor - view would be partial and then wrong if
>>>>>> allowed.
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> Romain Manni-Bucau
>>>>>> @rmannibucau
>>>>>> http://www.tomitribe.com
>>>>>> http://rmannibucau.wordpress.com
>>>>>> https://github.com/rmannibucau
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 2015-01-01 11:18 GMT+01:00 Anatole Tresch <at...@gmail.com>:
>>>>>>> I would say lazyness for evaluation is a must (due to possible
>>>>> environment
>>>>>>> dependencies properties returned can be different!), caching of
>>>>> aggregated
>>>>>>> configuration on some level comes on top. Or did I get something
>>>>> wrong...?
>>>>>>> 
>>>>>>> 
>>>>>>> 2015-01-01 11:11 GMT+01:00 Romain Manni-Bucau
>>>>>> <rm...@gmail.com>:
>>>>>>> 
>>>>>>>> wouldnt it make it harder to implement - ie not user friendly but
>>> impl
>>>>>>>> friendly only?
>>>>>>>> 
>>>>>>>> + not sure lazyness is useful we need an aggregated config anyway
>>> (=
>>>>>> eager)
>>>>>>>> 
>>>>>>>> 
>>>>>>>> Romain Manni-Bucau
>>>>>>>> @rmannibucau
>>>>>>>> http://www.tomitribe.com
>>>>>>>> http://rmannibucau.wordpress.com
>>>>>>>> https://github.com/rmannibucau
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 2015-01-01 11:08 GMT+01:00 Anatole Tresch <an...@apache.org>:
>>>>>>>>> Hi all
>>>>>>>>> 
>>>>>>>>> another alternative that comes to my mind, would be to model
>> the
>>>>>>>> PropertySource
>>>>>>>>> more as a Stream<Map.Entry<String,String>> than a
>>>>>> provider:
>>>>>>>>> 
>>>>>>>>>   /**
>>>>>>>>>     * <p>Evaluate the current properties as Map, based on
>>>>>> the current
>>>>>>>>> defined properties. The resulting Map may not        * return
>> all
>>>>>> items
>>>>>>>>> accessible, when the PropertySource is filtering values for the
>>>>>> context,
>>>>>>>>> where the
>>>>>>>>>     * PropertySource is available.</p>
>>>>>>>>>     * <p>PropertySources that are not filtering data,
>>>>>> should either add
>>>>>>>> or
>>>>>>>>> override entries, but not remove
>>>>>>>>>     * any entries provided by the input map.
>>>>>>>>>     *
>>>>>>>>>     * @return the a corresponding map, never null.
>>>>>>>>>     * //X TODO or should we just do getPropertyKeys()? Think
>>>>>> about
>>>>>>>>> security (key) vs easier merging (full map)?
>>>>>>>>>     */
>>>>>>>>>    Stream<Map.Entry<String,String>> getProperties(
>>>>>>>>> Stream<Map.Entry<String,String>>
>>>>>> currentEntriesStream);
>>>>>>>>> 
>>>>>>>>> This allows us to export all collecting and filtering
>> activities
>>>>>> into the
>>>>>>>>> Stream API, and also adding entries is simple:
>>>>>>>>> 
>>>>>>>>> Stream stream = Stream.concat(stream1, stream2);
>>>>>>>>> 
>>>>>>>>> Our code gets bare simple and we profit from all the lazy
>>>>>> evaluation
>>>>>>>>> features....
>>>>>>>>> 
>>>>>>>>> WDYT?
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 2015-01-01 10:44 GMT+01:00 Anatole Tresch
>>>>>> <an...@apache.org>:
>>>>>>>>> 
>>>>>>>>>> Hi all
>>>>>>>>>> 
>>>>>>>>>> I would like to make a *SPI proposal* for PropertyFilter (on
>>>>>> API side we
>>>>>>>>>> have UnaryOperator<Configuration>). Hereby filters have
>>>>>> ordinals and
>>>>>>>>>> undergo the same ordering mechanism as PropertySources.
>>>>>> Filters get
>>>>>>>>>> similarly added to chain of evaluation as PropertySources
>>>>>>>> (implementations
>>>>>>>>>> must add @Priority -> do we need a programmatic
>>>>>> getOrdinal() to have a
>>>>>>>> more
>>>>>>>>>> concise usae here?), with one exceptional rule:
>>>>>>>>>> If a filter accidentally has the same ordinal as some
>>>>>> PropertySources in
>>>>>>>>>> the chain, filters are always executed AFTER PropertySources
>>>>>>>>>> 
>>>>>>>>>> So here is the interface:.
>>>>>>>>>> 
>>>>>>>>>> /**
>>>>>>>>>> * <p>Interface for filtering the current map of
>>>>>> properties during the
>>>>>>>>>> evaluation of the chain of PropertySources.
>>>>>>>>>> * Filters can be registered using the {@link
>>>>>>>>>> org.apache.tamaya.spi.ServiceContext} or added
>>>>>> programmatically
>>>>>>>>>> * by calling {@link
>>>>>>>>>> 
>>>>>> org.apache.tamaya.spi.ConfigurationContext#addPropertyFilter(int,
>>>>>>>>>> PropertyFilter)}. The ordinal
>>>>>>>>>> * hereby is defined by the corresponding {@code
>>>>>> @Priority}
>>>>>>>> annotation.</p>
>>>>>>>>>> * <p>Filters </p>
>>>>>>>>>> */
>>>>>>>>>> public interface PropertyFilter<T>{
>>>>>>>>>> 
>>>>>>>>>>    /**
>>>>>>>>>>     * Method to check, if a filter is available. Filters that
>>>>>> are
>>>>>>>>>> currently not available are not considered, when
>>>>>>>>>>     * evaluating the current chain of {@link
>>>>>>>>>> org.apache.tamaya.spi.PropertySource} instances.
>>>>>>>>>>     * @return true, if a filter is available.
>>>>>>>>>>     */
>>>>>>>>>>    public boolean isAvailable();
>>>>>>>>>> 
>>>>>>>>>>    /**
>>>>>>>>>>     * Maps the current properties map to the filtered version
>>>>>> of the
>>>>>>>> map.
>>>>>>>>>> The resulting map will be used as the input
>>>>>>>>>>     * for all subsequent handling of PropertySources in the
>>>>>>>>>> PropertySource chain.
>>>>>>>>>>     * @param currentMap the current property map, not null
>>>>>>>>>>     * @return the filtered map, never null.
>>>>>>>>>>     */
>>>>>>>>>>    Map<String,String> map(Map<String,String>
>>>>>> currentMap);
>>>>>>>>>> 
>>>>>>>>>> }
>>>>>>>>>> 
>>>>>>>>>> WDYT?
>>>>>>>>>> 
>>>>>>>>>> Anatole
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 2014-12-29 17:59 GMT+01:00 Anatole Tresch
>>>>>> <an...@apache.org>:
>>>>>>>>>> 
>>>>>>>>>>> Whereas overrides are simple with this mechanism. I am
>>>>>> asking how we
>>>>>>>> want
>>>>>>>>>>> to do other aggregation types, or filtering of values.
>>>>>>>>>>> 
>>>>>>>>>>> *On the API side* filtering is always simple, since we
>>>>>> have the
>>>>>>>>>>> functional extension points
>>>>>> (UnaryOperator<Configuration>), where we
>>>>>>>> can
>>>>>>>>>>> easily filter the whole configuration, and the adapters
>>>>>> for filtering
>>>>>>>>>>> single entries on access.
>>>>>>>>>>> 
>>>>>>>>>>> But let's assume the following use case:
>>>>>>>>>>> 
>>>>>>>>>>> *We have a system low level configuration S, which also
>>>>>> contains a few
>>>>>>>>>>> entries, I do not want to see on application config A.
>>>>>> Given our
>>>>>>>> priority
>>>>>>>>>>> based assembly this is not possible, since I only can
>>>>>> override the
>>>>>>>> values
>>>>>>>>>>> from my parent sources, but never remove them.*
>>>>>>>>>>> 
>>>>>>>>>>> Thinking on this the following ideas come to my mind:
>>>>>>>>>>> 
>>>>>>>>>>> 1) *add PropertySource filters to my chain. I could use
>>>>>> the same
>>>>>>>>>>> priorities. The bad thing is that it is not clear if the
>>>>>> filter should
>>>>>>>> be
>>>>>>>>>>> active also, when no source with a similar priority is
>>>>>> active and if
>>>>>>>> so if
>>>>>>>>>>> it should be before or after, so this will not work.*
>>>>>>>>>>> *2) I could define a PropertySource to be filtering, so it
>>>>>> is able not
>>>>>>>>>>> only to override, but also to remove entries. This
>>>>>> "filter" can be
>>>>>>>>>>> activated or deactivated depending on the current context.
>>>>>> Given that
>>>>>>>> (if
>>>>>>>>>>> we have set a correct priority value), all works fine.*
>>>>>>>>>>> *3) I could define a separate filter layer which is
>>>>>> allowed to filter
>>>>>>>> the
>>>>>>>>>>> entries, before they are published (after all overrides
>>>>>> were done).
>>>>>>>> This
>>>>>>>>>>> would probably be transparently be hooked into the
>>>>>> "Configuration
>>>>>>>> Layer",
>>>>>>>>>>> which takes the current property sources and creates the
>>>>>> final map of
>>>>>>>>>>> properties. Then on the final Map the filter service is
>>>>>> operating
>>>>>>>> before
>>>>>>>>>>> any data is handed out to the callers.*
>>>>>>>>>>> 
>>>>>>>>>>> I think (3) is the best solution here. Nevertheless we
>>>>>> should think on
>>>>>>>>>>> how this can be done the best way and how the SPI here
>>>>>> would look like.
>>>>>>>>>>> Also I think, we might have several filters active, e.g.
>>>>>> one for
>>>>>>>> logging,
>>>>>>>>>>> one for admins, one which considers the current user's
>>>>>> role (aka
>>>>>>>> views), ...
>>>>>>>>>>> 
>>>>>>>>>>> So ideas are welcome ;)
>>>>>>>>>>> 
>>>>>>>>>>> Best,
>>>>>>>>>>> Anatole
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> --
>>>>>>> *Anatole Tresch*
>>>>>>> Java Engineer & Architect, JSR Spec Lead
>>>>>>> Glärnischweg 10
>>>>>>> CH - 8620 Wetzikon
>>>>>>> 
>>>>>>> *Switzerland, Europe Zurich, GMT+1*
>>>>>>> *Twitter:  @atsticks*
>>>>>>> *Blogs: **http://javaremarkables.blogspot.ch/
>>>>>>> <http://javaremarkables.blogspot.ch/>*
>>>>>>> 
>>>>>>> *Google: atsticksMobile  +41-76 344 62 79*
>>>>>> 
>>>>> 
>>>> 
>>>> 
>>>> 
>>>> --
>>>> *Anatole Tresch*
>>>> Java Engineer & Architect, JSR Spec Lead
>>>> Glärnischweg 10
>>>> CH - 8620 Wetzikon
>>>> 
>>>> *Switzerland, Europe Zurich, GMT+1*
>>>> *Twitter:  @atsticks*
>>>> *Blogs: **http://javaremarkables.blogspot.ch/
>>>> <http://javaremarkables.blogspot.ch/>*
>>>> 
>>>> *Google: atsticksMobile  +41-76 344 62 79*
>>> 
>> 

Re: PropertyFiltering

[Romain Manni-Bucau <rm...@gmail.com>]

I wouldnt start on a tree solution and i dont agree. Aggregation is fine
and the most performant solution, only issue is not scannable sources and
we can say we dont support them IMO.
Le 1 janv. 2015 20:22, "Anatole Tresch" <at...@gmail.com> a écrit :

> Hi Romain
>
> Not necessarily, but you simply do mot want to see everything everywhere.
> You config needs on domain level are different than on app level. You
> simply do not want to see sll the low level grap. Since we do not have
> named configurations anymore, we need true filtering at the levels spi
> implementors want it to have.
> Furthernore: evaluating a value with the current override mechanism is too
> costly becausr you are NOT using your chain in an efficient way: when you
> have 2 sources a and b. And b defines a value, you always have to evaluate
> b, though the fact that b overrides a, so is not relevant for that case.
> Similarly evaluating a non scannable value from a is easy. Just b must now,
> which values are not cacheable. That coulb be done by some meta data, ie
> with another entry compliant with some metadata scheme.
>
> So you get everything: efficiency , filtering, caching, contextuality...
> The only thing that this works is to provide access to the current
> evaluation chain eg some evaluation context:
> Something like:
>
> Interface evslctx{
>
> Map<string,string> getCurrentProperties();
> String getCurrentValue();
> Boolean contextHasChanged();
> }
>
> Hereby context has changed can be used to flag a change in the preceedong
> sources , so the current source can react on the change, eg by reevaluating
> getCurrentProperties();
>
> Within the ConfigContext we then can access the topmost propertysource
> available (therefore available and empty are not the same!). On first
> sccess it will go down the gull chsin, whereas later this might pnly be
> necessary when needed...
>
> Anatole
>
> Romain Manni-Bucau <rm...@gmail.com> schrieb am Do., 1. Jan. 2015 um
> 19:46:
>
> > Do you mean we loose the aggregation? Sounds like loosing the
> > configuration management/overview then, are you sure it is good?
> >
> > About filtering I would just care about modifying filters and ignore
> > logging ones = tamaya shouldnt log any value (can log its source if we
> > need) IMO. This logging layer is a very particular need and each app
> > know if a value can be logged or not so adding an abstraction which is
> > not needed in practise seems like overcomplicated to me.
> >
> >
> > Romain Manni-Bucau
> > @rmannibucau
> > http://www.tomitribe.com
> > http://rmannibucau.wordpress.com
> > https://github.com/rmannibucau
> >
> >
> > 2015-01-01 14:18 GMT+01:00 Anatole Tresch <at...@gmail.com>:
> > > See inline...
> > >
> > > 2015-01-01 12:57 GMT+01:00 Mark Struberg <st...@yahoo.de>:
> > >
> > >> If lazy or eager is a matter of each specific Filter. There is no
> common
> > >> answer to that.
> > >>
> > >> The only reason why we have a problem with it is that the the proposed
> > >> interface handles the whole map.
> > >
> > > If we would instead just apply the filtering chain on single attributes
> > >> (while they get queried) then all those problems would vanish.
> > >>
> > >
> > > Given what you said above. its an implementation aspect
> > > of the filter
> > > , if a filter passes on a key accessed to the chain for reevaluation,
> > > whereas caching others we have exactly that selective filtering
> > > ... so having control where a filter is placed within the chain of
> > > property sources for me is the crucial aspect. I also think for the
> sake
> > of
> > > simplicity filters should have the same granularity as property
> sources,
> > > e.g.
> > >
> > > envprops -> sysprops -> domain-cfg -> domainFilter -> app-cfg ->
> > > securityFilter
> > >
> > > Thats the reason why I started to think to model property source and
> > > filters very similarly, or even model the property source as property
> > chain
> > > member mapping the current input properties to the output properties to
> > > pass to the next member (in that case PropertySource may also undergo a
> > > renaming).
> > > Alternatively we could model the property chain also the other way
> round.
> > > Instead of aggregating all up until we end up with a result, we may
> > simply
> > > ask the highest available (therefore you need an availability flag)
> > > PropertySource for the current configuration. This implementation then
> > can
> > > deliberately decide, if it can provide a key, or it has it to delegate
> to
> > > its parent PropertySource:
> > >
> > > Map<String,String> getProperties(PropertySource parentPropertySource);
> > > Optional<String> get(String key,
> > > PropertySource parentPropertySource);
> > >
> > > This would solve all: implementation, filtering, caching and still the
> > > ConfigurationContext
> > > around has control over the current chain (he passes the parent
> source)..
> > >
> > > There is btw also a difference between filtering to change values (e.g.
> > >> decrypt ciphered passwords) and filtering for the purpose of logging.
> > >>
> > > Agreed. I think should try to list/discuss the use cases in more
> detail,
> > > when and how configuration should be logged. In most cases I would
> assume
> > > that we can filter by just adding a top level filter, before passing
> the
> > > config to the logger...
> > >
> > >
> > >
> > >>
> > >> LieGrue,
> > >> strub
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >> > On Thursday, 1 January 2015, 11:25, Romain Manni-Bucau <
> > >> rmannibucau@gmail.com> wrote:
> > >> > > I think we need both but eager initialization should be the
> default
> > >> > (isScannable = true). All other usages (isScannable = false) should
> be
> > >> > lazy and used with caution (main case I have in mind is JNDI and I
> > >> > wouldnt rely on it really today).
> > >> >
> > >> > What I did in my common impl of conf was to build a Map<String,
> > >> > String> then "sources" were able to trigger a refresh event making
> > >> > the
> > >> > configuration rebuilding the Map then do a diff and switch if
> > >> > different. This implies only eager eval and makes conf management
> very
> > >> > easy but still allowing updates. Only case it doesnt handle is lazy
> > >> > eval (JNDI) but this was intended cause harder to
> > >> > manage/control/monitor - view would be partial and then wrong if
> > >> > allowed.
> > >> >
> > >> >
> > >> >
> > >> > Romain Manni-Bucau
> > >> > @rmannibucau
> > >> > http://www.tomitribe.com
> > >> > http://rmannibucau.wordpress.com
> > >> > https://github.com/rmannibucau
> > >> >
> > >> >
> > >> >
> > >> > 2015-01-01 11:18 GMT+01:00 Anatole Tresch <at...@gmail.com>:
> > >> >>  I would say lazyness for evaluation is a must (due to possible
> > >> environment
> > >> >>  dependencies properties returned can be different!), caching of
> > >> aggregated
> > >> >>  configuration on some level comes on top. Or did I get something
> > >> wrong...?
> > >> >>
> > >> >>
> > >> >>  2015-01-01 11:11 GMT+01:00 Romain Manni-Bucau
> > >> > <rm...@gmail.com>:
> > >> >>
> > >> >>>  wouldnt it make it harder to implement - ie not user friendly but
> > impl
> > >> >>>  friendly only?
> > >> >>>
> > >> >>>  + not sure lazyness is useful we need an aggregated config anyway
> > (=
> > >> > eager)
> > >> >>>
> > >> >>>
> > >> >>>  Romain Manni-Bucau
> > >> >>>  @rmannibucau
> > >> >>>  http://www.tomitribe.com
> > >> >>>  http://rmannibucau.wordpress.com
> > >> >>>  https://github.com/rmannibucau
> > >> >>>
> > >> >>>
> > >> >>>  2015-01-01 11:08 GMT+01:00 Anatole Tresch <an...@apache.org>:
> > >> >>>  > Hi all
> > >> >>>  >
> > >> >>>  > another alternative that comes to my mind, would be to model
> the
> > >> >>>  PropertySource
> > >> >>>  > more as a Stream<Map.Entry<String,String>> than a
> > >> > provider:
> > >> >>>  >
> > >> >>>  >    /**
> > >> >>>  >      * <p>Evaluate the current properties as Map, based on
> > >> > the current
> > >> >>>  > defined properties. The resulting Map may not        * return
> all
> > >> > items
> > >> >>>  > accessible, when the PropertySource is filtering values for the
> > >> > context,
> > >> >>>  > where the
> > >> >>>  >      * PropertySource is available.</p>
> > >> >>>  >      * <p>PropertySources that are not filtering data,
> > >> > should either add
> > >> >>>  or
> > >> >>>  > override entries, but not remove
> > >> >>>  >      * any entries provided by the input map.
> > >> >>>  >      *
> > >> >>>  >      * @return the a corresponding map, never null.
> > >> >>>  >      * //X TODO or should we just do getPropertyKeys()? Think
> > >> > about
> > >> >>>  > security (key) vs easier merging (full map)?
> > >> >>>  >      */
> > >> >>>  >     Stream<Map.Entry<String,String>> getProperties(
> > >> >>>  > Stream<Map.Entry<String,String>>
> > >> > currentEntriesStream);
> > >> >>>  >
> > >> >>>  > This allows us to export all collecting and filtering
> activities
> > >> > into the
> > >> >>>  > Stream API, and also adding entries is simple:
> > >> >>>  >
> > >> >>>  > Stream stream = Stream.concat(stream1, stream2);
> > >> >>>  >
> > >> >>>  > Our code gets bare simple and we profit from all the lazy
> > >> > evaluation
> > >> >>>  > features....
> > >> >>>  >
> > >> >>>  > WDYT?
> > >> >>>  >
> > >> >>>  >
> > >> >>>  >
> > >> >>>  > 2015-01-01 10:44 GMT+01:00 Anatole Tresch
> > >> > <an...@apache.org>:
> > >> >>>  >
> > >> >>>  >> Hi all
> > >> >>>  >>
> > >> >>>  >> I would like to make a *SPI proposal* for PropertyFilter (on
> > >> > API side we
> > >> >>>  >> have UnaryOperator<Configuration>). Hereby filters have
> > >> > ordinals and
> > >> >>>  >> undergo the same ordering mechanism as PropertySources.
> > >> > Filters get
> > >> >>>  >> similarly added to chain of evaluation as PropertySources
> > >> >>>  (implementations
> > >> >>>  >> must add @Priority -> do we need a programmatic
> > >> > getOrdinal() to have a
> > >> >>>  more
> > >> >>>  >> concise usae here?), with one exceptional rule:
> > >> >>>  >> If a filter accidentally has the same ordinal as some
> > >> > PropertySources in
> > >> >>>  >> the chain, filters are always executed AFTER PropertySources
> > >> >>>  >>
> > >> >>>  >> So here is the interface:.
> > >> >>>  >>
> > >> >>>  >> /**
> > >> >>>  >>  * <p>Interface for filtering the current map of
> > >> > properties during the
> > >> >>>  >> evaluation of the chain of PropertySources.
> > >> >>>  >>  * Filters can be registered using the {@link
> > >> >>>  >> org.apache.tamaya.spi.ServiceContext} or added
> > >> > programmatically
> > >> >>>  >>  * by calling {@link
> > >> >>>  >>
> > >> > org.apache.tamaya.spi.ConfigurationContext#addPropertyFilter(int,
> > >> >>>  >> PropertyFilter)}. The ordinal
> > >> >>>  >>  * hereby is defined by the corresponding {@code
> > >> > @Priority}
> > >> >>>  annotation.</p>
> > >> >>>  >>  * <p>Filters </p>
> > >> >>>  >>  */
> > >> >>>  >> public interface PropertyFilter<T>{
> > >> >>>  >>
> > >> >>>  >>     /**
> > >> >>>  >>      * Method to check, if a filter is available. Filters that
> > >> > are
> > >> >>>  >> currently not available are not considered, when
> > >> >>>  >>      * evaluating the current chain of {@link
> > >> >>>  >> org.apache.tamaya.spi.PropertySource} instances.
> > >> >>>  >>      * @return true, if a filter is available.
> > >> >>>  >>      */
> > >> >>>  >>     public boolean isAvailable();
> > >> >>>  >>
> > >> >>>  >>     /**
> > >> >>>  >>      * Maps the current properties map to the filtered version
> > >> > of the
> > >> >>>  map.
> > >> >>>  >> The resulting map will be used as the input
> > >> >>>  >>      * for all subsequent handling of PropertySources in the
> > >> >>>  >> PropertySource chain.
> > >> >>>  >>      * @param currentMap the current property map, not null
> > >> >>>  >>      * @return the filtered map, never null.
> > >> >>>  >>      */
> > >> >>>  >>     Map<String,String> map(Map<String,String>
> > >> > currentMap);
> > >> >>>  >>
> > >> >>>  >> }
> > >> >>>  >>
> > >> >>>  >> WDYT?
> > >> >>>  >>
> > >> >>>  >> Anatole
> > >> >>>  >>
> > >> >>>  >>
> > >> >>>  >> 2014-12-29 17:59 GMT+01:00 Anatole Tresch
> > >> > <an...@apache.org>:
> > >> >>>  >>
> > >> >>>  >>> Whereas overrides are simple with this mechanism. I am
> > >> > asking how we
> > >> >>>  want
> > >> >>>  >>> to do other aggregation types, or filtering of values.
> > >> >>>  >>>
> > >> >>>  >>> *On the API side* filtering is always simple, since we
> > >> > have the
> > >> >>>  >>> functional extension points
> > >> > (UnaryOperator<Configuration>), where we
> > >> >>>  can
> > >> >>>  >>> easily filter the whole configuration, and the adapters
> > >> > for filtering
> > >> >>>  >>> single entries on access.
> > >> >>>  >>>
> > >> >>>  >>> But let's assume the following use case:
> > >> >>>  >>>
> > >> >>>  >>> *We have a system low level configuration S, which also
> > >> > contains a few
> > >> >>>  >>> entries, I do not want to see on application config A.
> > >> > Given our
> > >> >>>  priority
> > >> >>>  >>> based assembly this is not possible, since I only can
> > >> > override the
> > >> >>>  values
> > >> >>>  >>> from my parent sources, but never remove them.*
> > >> >>>  >>>
> > >> >>>  >>> Thinking on this the following ideas come to my mind:
> > >> >>>  >>>
> > >> >>>  >>> 1) *add PropertySource filters to my chain. I could use
> > >> > the same
> > >> >>>  >>> priorities. The bad thing is that it is not clear if the
> > >> > filter should
> > >> >>>  be
> > >> >>>  >>> active also, when no source with a similar priority is
> > >> > active and if
> > >> >>>  so if
> > >> >>>  >>> it should be before or after, so this will not work.*
> > >> >>>  >>> *2) I could define a PropertySource to be filtering, so it
> > >> > is able not
> > >> >>>  >>> only to override, but also to remove entries. This
> > >> > "filter" can be
> > >> >>>  >>> activated or deactivated depending on the current context.
> > >> > Given that
> > >> >>>  (if
> > >> >>>  >>> we have set a correct priority value), all works fine.*
> > >> >>>  >>> *3) I could define a separate filter layer which is
> > >> > allowed to filter
> > >> >>>  the
> > >> >>>  >>> entries, before they are published (after all overrides
> > >> > were done).
> > >> >>>  This
> > >> >>>  >>> would probably be transparently be hooked into the
> > >> > "Configuration
> > >> >>>  Layer",
> > >> >>>  >>> which takes the current property sources and creates the
> > >> > final map of
> > >> >>>  >>> properties. Then on the final Map the filter service is
> > >> > operating
> > >> >>>  before
> > >> >>>  >>> any data is handed out to the callers.*
> > >> >>>  >>>
> > >> >>>  >>> I think (3) is the best solution here. Nevertheless we
> > >> > should think on
> > >> >>>  >>> how this can be done the best way and how the SPI here
> > >> > would look like.
> > >> >>>  >>> Also I think, we might have several filters active, e.g.
> > >> > one for
> > >> >>>  logging,
> > >> >>>  >>> one for admins, one which considers the current user's
> > >> > role (aka
> > >> >>>  views), ...
> > >> >>>  >>>
> > >> >>>  >>> So ideas are welcome ;)
> > >> >>>  >>>
> > >> >>>  >>> Best,
> > >> >>>  >>> Anatole
> > >> >>>  >>>
> > >> >>>  >>>
> > >> >>>  >>
> > >> >>>
> > >> >>
> > >> >>
> > >> >>
> > >> >>  --
> > >> >>  *Anatole Tresch*
> > >> >>  Java Engineer & Architect, JSR Spec Lead
> > >> >>  Glärnischweg 10
> > >> >>  CH - 8620 Wetzikon
> > >> >>
> > >> >>  *Switzerland, Europe Zurich, GMT+1*
> > >> >>  *Twitter:  @atsticks*
> > >> >>  *Blogs: **http://javaremarkables.blogspot.ch/
> > >> >>  <http://javaremarkables.blogspot.ch/>*
> > >> >>
> > >> >>  *Google: atsticksMobile  +41-76 344 62 79*
> > >> >
> > >>
> > >
> > >
> > >
> > > --
> > > *Anatole Tresch*
> > > Java Engineer & Architect, JSR Spec Lead
> > > Glärnischweg 10
> > > CH - 8620 Wetzikon
> > >
> > > *Switzerland, Europe Zurich, GMT+1*
> > > *Twitter:  @atsticks*
> > > *Blogs: **http://javaremarkables.blogspot.ch/
> > > <http://javaremarkables.blogspot.ch/>*
> > >
> > > *Google: atsticksMobile  +41-76 344 62 79*
> >
>

Re: PropertyFiltering

[Anatole Tresch <at...@gmail.com>]

Hi Romain

Not necessarily, but you simply do mot want to see everything everywhere.
You config needs on domain level are different than on app level. You
simply do not want to see sll the low level grap. Since we do not have
named configurations anymore, we need true filtering at the levels spi
implementors want it to have.
Furthernore: evaluating a value with the current override mechanism is too
costly becausr you are NOT using your chain in an efficient way: when you
have 2 sources a and b. And b defines a value, you always have to evaluate
b, though the fact that b overrides a, so is not relevant for that case.
Similarly evaluating a non scannable value from a is easy. Just b must now,
which values are not cacheable. That coulb be done by some meta data, ie
with another entry compliant with some metadata scheme.

So you get everything: efficiency , filtering, caching, contextuality...
The only thing that this works is to provide access to the current
evaluation chain eg some evaluation context:
Something like:

Interface evslctx{

Map<string,string> getCurrentProperties();
String getCurrentValue();
Boolean contextHasChanged();
}

Hereby context has changed can be used to flag a change in the preceedong
sources , so the current source can react on the change, eg by reevaluating
getCurrentProperties();

Within the ConfigContext we then can access the topmost propertysource
available (therefore available and empty are not the same!). On first
sccess it will go down the gull chsin, whereas later this might pnly be
necessary when needed...

Anatole

Romain Manni-Bucau <rm...@gmail.com> schrieb am Do., 1. Jan. 2015 um
19:46:

> Do you mean we loose the aggregation? Sounds like loosing the
> configuration management/overview then, are you sure it is good?
>
> About filtering I would just care about modifying filters and ignore
> logging ones = tamaya shouldnt log any value (can log its source if we
> need) IMO. This logging layer is a very particular need and each app
> know if a value can be logged or not so adding an abstraction which is
> not needed in practise seems like overcomplicated to me.
>
>
> Romain Manni-Bucau
> @rmannibucau
> http://www.tomitribe.com
> http://rmannibucau.wordpress.com
> https://github.com/rmannibucau
>
>
> 2015-01-01 14:18 GMT+01:00 Anatole Tresch <at...@gmail.com>:
> > See inline...
> >
> > 2015-01-01 12:57 GMT+01:00 Mark Struberg <st...@yahoo.de>:
> >
> >> If lazy or eager is a matter of each specific Filter. There is no common
> >> answer to that.
> >>
> >> The only reason why we have a problem with it is that the the proposed
> >> interface handles the whole map.
> >
> > If we would instead just apply the filtering chain on single attributes
> >> (while they get queried) then all those problems would vanish.
> >>
> >
> > Given what you said above. its an implementation aspect
> > of the filter
> > , if a filter passes on a key accessed to the chain for reevaluation,
> > whereas caching others we have exactly that selective filtering
> > ... so having control where a filter is placed within the chain of
> > property sources for me is the crucial aspect. I also think for the sake
> of
> > simplicity filters should have the same granularity as property sources,
> > e.g.
> >
> > envprops -> sysprops -> domain-cfg -> domainFilter -> app-cfg ->
> > securityFilter
> >
> > Thats the reason why I started to think to model property source and
> > filters very similarly, or even model the property source as property
> chain
> > member mapping the current input properties to the output properties to
> > pass to the next member (in that case PropertySource may also undergo a
> > renaming).
> > Alternatively we could model the property chain also the other way round.
> > Instead of aggregating all up until we end up with a result, we may
> simply
> > ask the highest available (therefore you need an availability flag)
> > PropertySource for the current configuration. This implementation then
> can
> > deliberately decide, if it can provide a key, or it has it to delegate to
> > its parent PropertySource:
> >
> > Map<String,String> getProperties(PropertySource parentPropertySource);
> > Optional<String> get(String key,
> > PropertySource parentPropertySource);
> >
> > This would solve all: implementation, filtering, caching and still the
> > ConfigurationContext
> > around has control over the current chain (he passes the parent source)..
> >
> > There is btw also a difference between filtering to change values (e.g.
> >> decrypt ciphered passwords) and filtering for the purpose of logging.
> >>
> > Agreed. I think should try to list/discuss the use cases in more detail,
> > when and how configuration should be logged. In most cases I would assume
> > that we can filter by just adding a top level filter, before passing the
> > config to the logger...
> >
> >
> >
> >>
> >> LieGrue,
> >> strub
> >>
> >>
> >>
> >>
> >>
> >>
> >> > On Thursday, 1 January 2015, 11:25, Romain Manni-Bucau <
> >> rmannibucau@gmail.com> wrote:
> >> > > I think we need both but eager initialization should be the default
> >> > (isScannable = true). All other usages (isScannable = false) should be
> >> > lazy and used with caution (main case I have in mind is JNDI and I
> >> > wouldnt rely on it really today).
> >> >
> >> > What I did in my common impl of conf was to build a Map<String,
> >> > String> then "sources" were able to trigger a refresh event making
> >> > the
> >> > configuration rebuilding the Map then do a diff and switch if
> >> > different. This implies only eager eval and makes conf management very
> >> > easy but still allowing updates. Only case it doesnt handle is lazy
> >> > eval (JNDI) but this was intended cause harder to
> >> > manage/control/monitor - view would be partial and then wrong if
> >> > allowed.
> >> >
> >> >
> >> >
> >> > Romain Manni-Bucau
> >> > @rmannibucau
> >> > http://www.tomitribe.com
> >> > http://rmannibucau.wordpress.com
> >> > https://github.com/rmannibucau
> >> >
> >> >
> >> >
> >> > 2015-01-01 11:18 GMT+01:00 Anatole Tresch <at...@gmail.com>:
> >> >>  I would say lazyness for evaluation is a must (due to possible
> >> environment
> >> >>  dependencies properties returned can be different!), caching of
> >> aggregated
> >> >>  configuration on some level comes on top. Or did I get something
> >> wrong...?
> >> >>
> >> >>
> >> >>  2015-01-01 11:11 GMT+01:00 Romain Manni-Bucau
> >> > <rm...@gmail.com>:
> >> >>
> >> >>>  wouldnt it make it harder to implement - ie not user friendly but
> impl
> >> >>>  friendly only?
> >> >>>
> >> >>>  + not sure lazyness is useful we need an aggregated config anyway
> (=
> >> > eager)
> >> >>>
> >> >>>
> >> >>>  Romain Manni-Bucau
> >> >>>  @rmannibucau
> >> >>>  http://www.tomitribe.com
> >> >>>  http://rmannibucau.wordpress.com
> >> >>>  https://github.com/rmannibucau
> >> >>>
> >> >>>
> >> >>>  2015-01-01 11:08 GMT+01:00 Anatole Tresch <an...@apache.org>:
> >> >>>  > Hi all
> >> >>>  >
> >> >>>  > another alternative that comes to my mind, would be to model the
> >> >>>  PropertySource
> >> >>>  > more as a Stream<Map.Entry<String,String>> than a
> >> > provider:
> >> >>>  >
> >> >>>  >    /**
> >> >>>  >      * <p>Evaluate the current properties as Map, based on
> >> > the current
> >> >>>  > defined properties. The resulting Map may not        * return all
> >> > items
> >> >>>  > accessible, when the PropertySource is filtering values for the
> >> > context,
> >> >>>  > where the
> >> >>>  >      * PropertySource is available.</p>
> >> >>>  >      * <p>PropertySources that are not filtering data,
> >> > should either add
> >> >>>  or
> >> >>>  > override entries, but not remove
> >> >>>  >      * any entries provided by the input map.
> >> >>>  >      *
> >> >>>  >      * @return the a corresponding map, never null.
> >> >>>  >      * //X TODO or should we just do getPropertyKeys()? Think
> >> > about
> >> >>>  > security (key) vs easier merging (full map)?
> >> >>>  >      */
> >> >>>  >     Stream<Map.Entry<String,String>> getProperties(
> >> >>>  > Stream<Map.Entry<String,String>>
> >> > currentEntriesStream);
> >> >>>  >
> >> >>>  > This allows us to export all collecting and filtering activities
> >> > into the
> >> >>>  > Stream API, and also adding entries is simple:
> >> >>>  >
> >> >>>  > Stream stream = Stream.concat(stream1, stream2);
> >> >>>  >
> >> >>>  > Our code gets bare simple and we profit from all the lazy
> >> > evaluation
> >> >>>  > features....
> >> >>>  >
> >> >>>  > WDYT?
> >> >>>  >
> >> >>>  >
> >> >>>  >
> >> >>>  > 2015-01-01 10:44 GMT+01:00 Anatole Tresch
> >> > <an...@apache.org>:
> >> >>>  >
> >> >>>  >> Hi all
> >> >>>  >>
> >> >>>  >> I would like to make a *SPI proposal* for PropertyFilter (on
> >> > API side we
> >> >>>  >> have UnaryOperator<Configuration>). Hereby filters have
> >> > ordinals and
> >> >>>  >> undergo the same ordering mechanism as PropertySources.
> >> > Filters get
> >> >>>  >> similarly added to chain of evaluation as PropertySources
> >> >>>  (implementations
> >> >>>  >> must add @Priority -> do we need a programmatic
> >> > getOrdinal() to have a
> >> >>>  more
> >> >>>  >> concise usae here?), with one exceptional rule:
> >> >>>  >> If a filter accidentally has the same ordinal as some
> >> > PropertySources in
> >> >>>  >> the chain, filters are always executed AFTER PropertySources
> >> >>>  >>
> >> >>>  >> So here is the interface:.
> >> >>>  >>
> >> >>>  >> /**
> >> >>>  >>  * <p>Interface for filtering the current map of
> >> > properties during the
> >> >>>  >> evaluation of the chain of PropertySources.
> >> >>>  >>  * Filters can be registered using the {@link
> >> >>>  >> org.apache.tamaya.spi.ServiceContext} or added
> >> > programmatically
> >> >>>  >>  * by calling {@link
> >> >>>  >>
> >> > org.apache.tamaya.spi.ConfigurationContext#addPropertyFilter(int,
> >> >>>  >> PropertyFilter)}. The ordinal
> >> >>>  >>  * hereby is defined by the corresponding {@code
> >> > @Priority}
> >> >>>  annotation.</p>
> >> >>>  >>  * <p>Filters </p>
> >> >>>  >>  */
> >> >>>  >> public interface PropertyFilter<T>{
> >> >>>  >>
> >> >>>  >>     /**
> >> >>>  >>      * Method to check, if a filter is available. Filters that
> >> > are
> >> >>>  >> currently not available are not considered, when
> >> >>>  >>      * evaluating the current chain of {@link
> >> >>>  >> org.apache.tamaya.spi.PropertySource} instances.
> >> >>>  >>      * @return true, if a filter is available.
> >> >>>  >>      */
> >> >>>  >>     public boolean isAvailable();
> >> >>>  >>
> >> >>>  >>     /**
> >> >>>  >>      * Maps the current properties map to the filtered version
> >> > of the
> >> >>>  map.
> >> >>>  >> The resulting map will be used as the input
> >> >>>  >>      * for all subsequent handling of PropertySources in the
> >> >>>  >> PropertySource chain.
> >> >>>  >>      * @param currentMap the current property map, not null
> >> >>>  >>      * @return the filtered map, never null.
> >> >>>  >>      */
> >> >>>  >>     Map<String,String> map(Map<String,String>
> >> > currentMap);
> >> >>>  >>
> >> >>>  >> }
> >> >>>  >>
> >> >>>  >> WDYT?
> >> >>>  >>
> >> >>>  >> Anatole
> >> >>>  >>
> >> >>>  >>
> >> >>>  >> 2014-12-29 17:59 GMT+01:00 Anatole Tresch
> >> > <an...@apache.org>:
> >> >>>  >>
> >> >>>  >>> Whereas overrides are simple with this mechanism. I am
> >> > asking how we
> >> >>>  want
> >> >>>  >>> to do other aggregation types, or filtering of values.
> >> >>>  >>>
> >> >>>  >>> *On the API side* filtering is always simple, since we
> >> > have the
> >> >>>  >>> functional extension points
> >> > (UnaryOperator<Configuration>), where we
> >> >>>  can
> >> >>>  >>> easily filter the whole configuration, and the adapters
> >> > for filtering
> >> >>>  >>> single entries on access.
> >> >>>  >>>
> >> >>>  >>> But let's assume the following use case:
> >> >>>  >>>
> >> >>>  >>> *We have a system low level configuration S, which also
> >> > contains a few
> >> >>>  >>> entries, I do not want to see on application config A.
> >> > Given our
> >> >>>  priority
> >> >>>  >>> based assembly this is not possible, since I only can
> >> > override the
> >> >>>  values
> >> >>>  >>> from my parent sources, but never remove them.*
> >> >>>  >>>
> >> >>>  >>> Thinking on this the following ideas come to my mind:
> >> >>>  >>>
> >> >>>  >>> 1) *add PropertySource filters to my chain. I could use
> >> > the same
> >> >>>  >>> priorities. The bad thing is that it is not clear if the
> >> > filter should
> >> >>>  be
> >> >>>  >>> active also, when no source with a similar priority is
> >> > active and if
> >> >>>  so if
> >> >>>  >>> it should be before or after, so this will not work.*
> >> >>>  >>> *2) I could define a PropertySource to be filtering, so it
> >> > is able not
> >> >>>  >>> only to override, but also to remove entries. This
> >> > "filter" can be
> >> >>>  >>> activated or deactivated depending on the current context.
> >> > Given that
> >> >>>  (if
> >> >>>  >>> we have set a correct priority value), all works fine.*
> >> >>>  >>> *3) I could define a separate filter layer which is
> >> > allowed to filter
> >> >>>  the
> >> >>>  >>> entries, before they are published (after all overrides
> >> > were done).
> >> >>>  This
> >> >>>  >>> would probably be transparently be hooked into the
> >> > "Configuration
> >> >>>  Layer",
> >> >>>  >>> which takes the current property sources and creates the
> >> > final map of
> >> >>>  >>> properties. Then on the final Map the filter service is
> >> > operating
> >> >>>  before
> >> >>>  >>> any data is handed out to the callers.*
> >> >>>  >>>
> >> >>>  >>> I think (3) is the best solution here. Nevertheless we
> >> > should think on
> >> >>>  >>> how this can be done the best way and how the SPI here
> >> > would look like.
> >> >>>  >>> Also I think, we might have several filters active, e.g.
> >> > one for
> >> >>>  logging,
> >> >>>  >>> one for admins, one which considers the current user's
> >> > role (aka
> >> >>>  views), ...
> >> >>>  >>>
> >> >>>  >>> So ideas are welcome ;)
> >> >>>  >>>
> >> >>>  >>> Best,
> >> >>>  >>> Anatole
> >> >>>  >>>
> >> >>>  >>>
> >> >>>  >>
> >> >>>
> >> >>
> >> >>
> >> >>
> >> >>  --
> >> >>  *Anatole Tresch*
> >> >>  Java Engineer & Architect, JSR Spec Lead
> >> >>  Glärnischweg 10
> >> >>  CH - 8620 Wetzikon
> >> >>
> >> >>  *Switzerland, Europe Zurich, GMT+1*
> >> >>  *Twitter:  @atsticks*
> >> >>  *Blogs: **http://javaremarkables.blogspot.ch/
> >> >>  <http://javaremarkables.blogspot.ch/>*
> >> >>
> >> >>  *Google: atsticksMobile  +41-76 344 62 79*
> >> >
> >>
> >
> >
> >
> > --
> > *Anatole Tresch*
> > Java Engineer & Architect, JSR Spec Lead
> > Glärnischweg 10
> > CH - 8620 Wetzikon
> >
> > *Switzerland, Europe Zurich, GMT+1*
> > *Twitter:  @atsticks*
> > *Blogs: **http://javaremarkables.blogspot.ch/
> > <http://javaremarkables.blogspot.ch/>*
> >
> > *Google: atsticksMobile  +41-76 344 62 79*
>

Re: PropertyFiltering

[Romain Manni-Bucau <rm...@gmail.com>]

Do you mean we loose the aggregation? Sounds like loosing the
configuration management/overview then, are you sure it is good?

About filtering I would just care about modifying filters and ignore
logging ones = tamaya shouldnt log any value (can log its source if we
need) IMO. This logging layer is a very particular need and each app
know if a value can be logged or not so adding an abstraction which is
not needed in practise seems like overcomplicated to me.


Romain Manni-Bucau
@rmannibucau
http://www.tomitribe.com
http://rmannibucau.wordpress.com
https://github.com/rmannibucau


2015-01-01 14:18 GMT+01:00 Anatole Tresch <at...@gmail.com>:
> See inline...
>
> 2015-01-01 12:57 GMT+01:00 Mark Struberg <st...@yahoo.de>:
>
>> If lazy or eager is a matter of each specific Filter. There is no common
>> answer to that.
>>
>> The only reason why we have a problem with it is that the the proposed
>> interface handles the whole map.
>
> If we would instead just apply the filtering chain on single attributes
>> (while they get queried) then all those problems would vanish.
>>
>
> Given what you said above. its an implementation aspect
> of the filter
> , if a filter passes on a key accessed to the chain for reevaluation,
> whereas caching others we have exactly that selective filtering
> ... so having control where a filter is placed within the chain of
> property sources for me is the crucial aspect. I also think for the sake of
> simplicity filters should have the same granularity as property sources,
> e.g.
>
> envprops -> sysprops -> domain-cfg -> domainFilter -> app-cfg ->
> securityFilter
>
> Thats the reason why I started to think to model property source and
> filters very similarly, or even model the property source as property chain
> member mapping the current input properties to the output properties to
> pass to the next member (in that case PropertySource may also undergo a
> renaming).
> Alternatively we could model the property chain also the other way round.
> Instead of aggregating all up until we end up with a result, we may simply
> ask the highest available (therefore you need an availability flag)
> PropertySource for the current configuration. This implementation then can
> deliberately decide, if it can provide a key, or it has it to delegate to
> its parent PropertySource:
>
> Map<String,String> getProperties(PropertySource parentPropertySource);
> Optional<String> get(String key,
> PropertySource parentPropertySource);
>
> This would solve all: implementation, filtering, caching and still the
> ConfigurationContext
> around has control over the current chain (he passes the parent source)..
>
> There is btw also a difference between filtering to change values (e.g.
>> decrypt ciphered passwords) and filtering for the purpose of logging.
>>
> Agreed. I think should try to list/discuss the use cases in more detail,
> when and how configuration should be logged. In most cases I would assume
> that we can filter by just adding a top level filter, before passing the
> config to the logger...
>
>
>
>>
>> LieGrue,
>> strub
>>
>>
>>
>>
>>
>>
>> > On Thursday, 1 January 2015, 11:25, Romain Manni-Bucau <
>> rmannibucau@gmail.com> wrote:
>> > > I think we need both but eager initialization should be the default
>> > (isScannable = true). All other usages (isScannable = false) should be
>> > lazy and used with caution (main case I have in mind is JNDI and I
>> > wouldnt rely on it really today).
>> >
>> > What I did in my common impl of conf was to build a Map<String,
>> > String> then "sources" were able to trigger a refresh event making
>> > the
>> > configuration rebuilding the Map then do a diff and switch if
>> > different. This implies only eager eval and makes conf management very
>> > easy but still allowing updates. Only case it doesnt handle is lazy
>> > eval (JNDI) but this was intended cause harder to
>> > manage/control/monitor - view would be partial and then wrong if
>> > allowed.
>> >
>> >
>> >
>> > Romain Manni-Bucau
>> > @rmannibucau
>> > http://www.tomitribe.com
>> > http://rmannibucau.wordpress.com
>> > https://github.com/rmannibucau
>> >
>> >
>> >
>> > 2015-01-01 11:18 GMT+01:00 Anatole Tresch <at...@gmail.com>:
>> >>  I would say lazyness for evaluation is a must (due to possible
>> environment
>> >>  dependencies properties returned can be different!), caching of
>> aggregated
>> >>  configuration on some level comes on top. Or did I get something
>> wrong...?
>> >>
>> >>
>> >>  2015-01-01 11:11 GMT+01:00 Romain Manni-Bucau
>> > <rm...@gmail.com>:
>> >>
>> >>>  wouldnt it make it harder to implement - ie not user friendly but impl
>> >>>  friendly only?
>> >>>
>> >>>  + not sure lazyness is useful we need an aggregated config anyway (=
>> > eager)
>> >>>
>> >>>
>> >>>  Romain Manni-Bucau
>> >>>  @rmannibucau
>> >>>  http://www.tomitribe.com
>> >>>  http://rmannibucau.wordpress.com
>> >>>  https://github.com/rmannibucau
>> >>>
>> >>>
>> >>>  2015-01-01 11:08 GMT+01:00 Anatole Tresch <an...@apache.org>:
>> >>>  > Hi all
>> >>>  >
>> >>>  > another alternative that comes to my mind, would be to model the
>> >>>  PropertySource
>> >>>  > more as a Stream<Map.Entry<String,String>> than a
>> > provider:
>> >>>  >
>> >>>  >    /**
>> >>>  >      * <p>Evaluate the current properties as Map, based on
>> > the current
>> >>>  > defined properties. The resulting Map may not        * return all
>> > items
>> >>>  > accessible, when the PropertySource is filtering values for the
>> > context,
>> >>>  > where the
>> >>>  >      * PropertySource is available.</p>
>> >>>  >      * <p>PropertySources that are not filtering data,
>> > should either add
>> >>>  or
>> >>>  > override entries, but not remove
>> >>>  >      * any entries provided by the input map.
>> >>>  >      *
>> >>>  >      * @return the a corresponding map, never null.
>> >>>  >      * //X TODO or should we just do getPropertyKeys()? Think
>> > about
>> >>>  > security (key) vs easier merging (full map)?
>> >>>  >      */
>> >>>  >     Stream<Map.Entry<String,String>> getProperties(
>> >>>  > Stream<Map.Entry<String,String>>
>> > currentEntriesStream);
>> >>>  >
>> >>>  > This allows us to export all collecting and filtering activities
>> > into the
>> >>>  > Stream API, and also adding entries is simple:
>> >>>  >
>> >>>  > Stream stream = Stream.concat(stream1, stream2);
>> >>>  >
>> >>>  > Our code gets bare simple and we profit from all the lazy
>> > evaluation
>> >>>  > features....
>> >>>  >
>> >>>  > WDYT?
>> >>>  >
>> >>>  >
>> >>>  >
>> >>>  > 2015-01-01 10:44 GMT+01:00 Anatole Tresch
>> > <an...@apache.org>:
>> >>>  >
>> >>>  >> Hi all
>> >>>  >>
>> >>>  >> I would like to make a *SPI proposal* for PropertyFilter (on
>> > API side we
>> >>>  >> have UnaryOperator<Configuration>). Hereby filters have
>> > ordinals and
>> >>>  >> undergo the same ordering mechanism as PropertySources.
>> > Filters get
>> >>>  >> similarly added to chain of evaluation as PropertySources
>> >>>  (implementations
>> >>>  >> must add @Priority -> do we need a programmatic
>> > getOrdinal() to have a
>> >>>  more
>> >>>  >> concise usae here?), with one exceptional rule:
>> >>>  >> If a filter accidentally has the same ordinal as some
>> > PropertySources in
>> >>>  >> the chain, filters are always executed AFTER PropertySources
>> >>>  >>
>> >>>  >> So here is the interface:.
>> >>>  >>
>> >>>  >> /**
>> >>>  >>  * <p>Interface for filtering the current map of
>> > properties during the
>> >>>  >> evaluation of the chain of PropertySources.
>> >>>  >>  * Filters can be registered using the {@link
>> >>>  >> org.apache.tamaya.spi.ServiceContext} or added
>> > programmatically
>> >>>  >>  * by calling {@link
>> >>>  >>
>> > org.apache.tamaya.spi.ConfigurationContext#addPropertyFilter(int,
>> >>>  >> PropertyFilter)}. The ordinal
>> >>>  >>  * hereby is defined by the corresponding {@code
>> > @Priority}
>> >>>  annotation.</p>
>> >>>  >>  * <p>Filters </p>
>> >>>  >>  */
>> >>>  >> public interface PropertyFilter<T>{
>> >>>  >>
>> >>>  >>     /**
>> >>>  >>      * Method to check, if a filter is available. Filters that
>> > are
>> >>>  >> currently not available are not considered, when
>> >>>  >>      * evaluating the current chain of {@link
>> >>>  >> org.apache.tamaya.spi.PropertySource} instances.
>> >>>  >>      * @return true, if a filter is available.
>> >>>  >>      */
>> >>>  >>     public boolean isAvailable();
>> >>>  >>
>> >>>  >>     /**
>> >>>  >>      * Maps the current properties map to the filtered version
>> > of the
>> >>>  map.
>> >>>  >> The resulting map will be used as the input
>> >>>  >>      * for all subsequent handling of PropertySources in the
>> >>>  >> PropertySource chain.
>> >>>  >>      * @param currentMap the current property map, not null
>> >>>  >>      * @return the filtered map, never null.
>> >>>  >>      */
>> >>>  >>     Map<String,String> map(Map<String,String>
>> > currentMap);
>> >>>  >>
>> >>>  >> }
>> >>>  >>
>> >>>  >> WDYT?
>> >>>  >>
>> >>>  >> Anatole
>> >>>  >>
>> >>>  >>
>> >>>  >> 2014-12-29 17:59 GMT+01:00 Anatole Tresch
>> > <an...@apache.org>:
>> >>>  >>
>> >>>  >>> Whereas overrides are simple with this mechanism. I am
>> > asking how we
>> >>>  want
>> >>>  >>> to do other aggregation types, or filtering of values.
>> >>>  >>>
>> >>>  >>> *On the API side* filtering is always simple, since we
>> > have the
>> >>>  >>> functional extension points
>> > (UnaryOperator<Configuration>), where we
>> >>>  can
>> >>>  >>> easily filter the whole configuration, and the adapters
>> > for filtering
>> >>>  >>> single entries on access.
>> >>>  >>>
>> >>>  >>> But let's assume the following use case:
>> >>>  >>>
>> >>>  >>> *We have a system low level configuration S, which also
>> > contains a few
>> >>>  >>> entries, I do not want to see on application config A.
>> > Given our
>> >>>  priority
>> >>>  >>> based assembly this is not possible, since I only can
>> > override the
>> >>>  values
>> >>>  >>> from my parent sources, but never remove them.*
>> >>>  >>>
>> >>>  >>> Thinking on this the following ideas come to my mind:
>> >>>  >>>
>> >>>  >>> 1) *add PropertySource filters to my chain. I could use
>> > the same
>> >>>  >>> priorities. The bad thing is that it is not clear if the
>> > filter should
>> >>>  be
>> >>>  >>> active also, when no source with a similar priority is
>> > active and if
>> >>>  so if
>> >>>  >>> it should be before or after, so this will not work.*
>> >>>  >>> *2) I could define a PropertySource to be filtering, so it
>> > is able not
>> >>>  >>> only to override, but also to remove entries. This
>> > "filter" can be
>> >>>  >>> activated or deactivated depending on the current context.
>> > Given that
>> >>>  (if
>> >>>  >>> we have set a correct priority value), all works fine.*
>> >>>  >>> *3) I could define a separate filter layer which is
>> > allowed to filter
>> >>>  the
>> >>>  >>> entries, before they are published (after all overrides
>> > were done).
>> >>>  This
>> >>>  >>> would probably be transparently be hooked into the
>> > "Configuration
>> >>>  Layer",
>> >>>  >>> which takes the current property sources and creates the
>> > final map of
>> >>>  >>> properties. Then on the final Map the filter service is
>> > operating
>> >>>  before
>> >>>  >>> any data is handed out to the callers.*
>> >>>  >>>
>> >>>  >>> I think (3) is the best solution here. Nevertheless we
>> > should think on
>> >>>  >>> how this can be done the best way and how the SPI here
>> > would look like.
>> >>>  >>> Also I think, we might have several filters active, e.g.
>> > one for
>> >>>  logging,
>> >>>  >>> one for admins, one which considers the current user's
>> > role (aka
>> >>>  views), ...
>> >>>  >>>
>> >>>  >>> So ideas are welcome ;)
>> >>>  >>>
>> >>>  >>> Best,
>> >>>  >>> Anatole
>> >>>  >>>
>> >>>  >>>
>> >>>  >>
>> >>>
>> >>
>> >>
>> >>
>> >>  --
>> >>  *Anatole Tresch*
>> >>  Java Engineer & Architect, JSR Spec Lead
>> >>  Glärnischweg 10
>> >>  CH - 8620 Wetzikon
>> >>
>> >>  *Switzerland, Europe Zurich, GMT+1*
>> >>  *Twitter:  @atsticks*
>> >>  *Blogs: **http://javaremarkables.blogspot.ch/
>> >>  <http://javaremarkables.blogspot.ch/>*
>> >>
>> >>  *Google: atsticksMobile  +41-76 344 62 79*
>> >
>>
>
>
>
> --
> *Anatole Tresch*
> Java Engineer & Architect, JSR Spec Lead
> Glärnischweg 10
> CH - 8620 Wetzikon
>
> *Switzerland, Europe Zurich, GMT+1*
> *Twitter:  @atsticks*
> *Blogs: **http://javaremarkables.blogspot.ch/
> <http://javaremarkables.blogspot.ch/>*
>
> *Google: atsticksMobile  +41-76 344 62 79*

Re: PropertyFiltering

[Anatole Tresch <at...@gmail.com>]

See inline...

2015-01-01 12:57 GMT+01:00 Mark Struberg <st...@yahoo.de>:

> If lazy or eager is a matter of each specific Filter. There is no common
> answer to that.
>
> The only reason why we have a problem with it is that the the proposed
> interface handles the whole map.

If we would instead just apply the filtering chain on single attributes
> (while they get queried) then all those problems would vanish.
>
​
​Given what you said above. its an implementation aspect
​ of the filter​
, if a filter passes on a key accessed to the chain for reevaluation​,
whereas caching others we have exactly that selective filtering
​...​ so having control where a filter is placed within the chain of
property sources for me is the crucial aspect. I also think for the sake of
simplicity filters should have the same granularity as property sources,
e.g.

envprops -> sysprops -> domain-cfg -> domainFilter -> app-cfg ->
securityFilter

Thats the reason why I started to think to model property source and
filters very similarly, or even model the property source as property chain
member mapping the current input properties to the output properties to
pass to the next member (in that case PropertySource may also undergo a
renaming).
Alternatively we could model the property chain also the other way round.
Instead of aggregating all up until we end up with a result, we may simply
ask the highest available (therefore you need an availability flag)
PropertySource for the current configuration. This implementation then can
deliberately decide, if it can provide a key, or it has it to delegate to
its parent PropertySource:

Map<String,String> getProperties(PropertySource parentPropertySource);
Optional<String> get(String key,
PropertySource parentPropertySource);

This would solve all: implementation, filtering, caching and still the
ConfigurationContext
around has control over the current chain (he passes the parent source)..

There is btw also a difference between filtering to change values (e.g.
> decrypt ciphered passwords) and filtering for the purpose of logging.
>
​Agreed. I think should try to list/discuss the use cases in more detail,
when and how​ configuration should be logged. In most cases I would assume
that we can filter by just adding a top level filter, before passing the
config to the logger...



>
> LieGrue,
> strub
>
>
>
>
>
>
> > On Thursday, 1 January 2015, 11:25, Romain Manni-Bucau <
> rmannibucau@gmail.com> wrote:
> > > I think we need both but eager initialization should be the default
> > (isScannable = true). All other usages (isScannable = false) should be
> > lazy and used with caution (main case I have in mind is JNDI and I
> > wouldnt rely on it really today).
> >
> > What I did in my common impl of conf was to build a Map<String,
> > String> then "sources" were able to trigger a refresh event making
> > the
> > configuration rebuilding the Map then do a diff and switch if
> > different. This implies only eager eval and makes conf management very
> > easy but still allowing updates. Only case it doesnt handle is lazy
> > eval (JNDI) but this was intended cause harder to
> > manage/control/monitor - view would be partial and then wrong if
> > allowed.
> >
> >
> >
> > Romain Manni-Bucau
> > @rmannibucau
> > http://www.tomitribe.com
> > http://rmannibucau.wordpress.com
> > https://github.com/rmannibucau
> >
> >
> >
> > 2015-01-01 11:18 GMT+01:00 Anatole Tresch <at...@gmail.com>:
> >>  I would say lazyness for evaluation is a must (due to possible
> environment
> >>  dependencies properties returned can be different!), caching of
> aggregated
> >>  configuration on some level comes on top. Or did I get something
> wrong...?
> >>
> >>
> >>  2015-01-01 11:11 GMT+01:00 Romain Manni-Bucau
> > <rm...@gmail.com>:
> >>
> >>>  wouldnt it make it harder to implement - ie not user friendly but impl
> >>>  friendly only?
> >>>
> >>>  + not sure lazyness is useful we need an aggregated config anyway (=
> > eager)
> >>>
> >>>
> >>>  Romain Manni-Bucau
> >>>  @rmannibucau
> >>>  http://www.tomitribe.com
> >>>  http://rmannibucau.wordpress.com
> >>>  https://github.com/rmannibucau
> >>>
> >>>
> >>>  2015-01-01 11:08 GMT+01:00 Anatole Tresch <an...@apache.org>:
> >>>  > Hi all
> >>>  >
> >>>  > another alternative that comes to my mind, would be to model the
> >>>  PropertySource
> >>>  > more as a Stream<Map.Entry<String,String>> than a
> > provider:
> >>>  >
> >>>  >    /**
> >>>  >      * <p>Evaluate the current properties as Map, based on
> > the current
> >>>  > defined properties. The resulting Map may not        * return all
> > items
> >>>  > accessible, when the PropertySource is filtering values for the
> > context,
> >>>  > where the
> >>>  >      * PropertySource is available.</p>
> >>>  >      * <p>PropertySources that are not filtering data,
> > should either add
> >>>  or
> >>>  > override entries, but not remove
> >>>  >      * any entries provided by the input map.
> >>>  >      *
> >>>  >      * @return the a corresponding map, never null.
> >>>  >      * //X TODO or should we just do getPropertyKeys()? Think
> > about
> >>>  > security (key) vs easier merging (full map)?
> >>>  >      */
> >>>  >     Stream<Map.Entry<String,String>> getProperties(
> >>>  > Stream<Map.Entry<String,String>>
> > currentEntriesStream);
> >>>  >
> >>>  > This allows us to export all collecting and filtering activities
> > into the
> >>>  > Stream API, and also adding entries is simple:
> >>>  >
> >>>  > Stream stream = Stream.concat(stream1, stream2);
> >>>  >
> >>>  > Our code gets bare simple and we profit from all the lazy
> > evaluation
> >>>  > features....
> >>>  >
> >>>  > WDYT?
> >>>  >
> >>>  >
> >>>  >
> >>>  > 2015-01-01 10:44 GMT+01:00 Anatole Tresch
> > <an...@apache.org>:
> >>>  >
> >>>  >> Hi all
> >>>  >>
> >>>  >> I would like to make a *SPI proposal* for PropertyFilter (on
> > API side we
> >>>  >> have UnaryOperator<Configuration>). Hereby filters have
> > ordinals and
> >>>  >> undergo the same ordering mechanism as PropertySources.
> > Filters get
> >>>  >> similarly added to chain of evaluation as PropertySources
> >>>  (implementations
> >>>  >> must add @Priority -> do we need a programmatic
> > getOrdinal() to have a
> >>>  more
> >>>  >> concise usae here?), with one exceptional rule:
> >>>  >> If a filter accidentally has the same ordinal as some
> > PropertySources in
> >>>  >> the chain, filters are always executed AFTER PropertySources
> >>>  >>
> >>>  >> So here is the interface:.
> >>>  >>
> >>>  >> /**
> >>>  >>  * <p>Interface for filtering the current map of
> > properties during the
> >>>  >> evaluation of the chain of PropertySources.
> >>>  >>  * Filters can be registered using the {@link
> >>>  >> org.apache.tamaya.spi.ServiceContext} or added
> > programmatically
> >>>  >>  * by calling {@link
> >>>  >>
> > org.apache.tamaya.spi.ConfigurationContext#addPropertyFilter(int,
> >>>  >> PropertyFilter)}. The ordinal
> >>>  >>  * hereby is defined by the corresponding {@code
> > @Priority}
> >>>  annotation.</p>
> >>>  >>  * <p>Filters </p>
> >>>  >>  */
> >>>  >> public interface PropertyFilter<T>{
> >>>  >>
> >>>  >>     /**
> >>>  >>      * Method to check, if a filter is available. Filters that
> > are
> >>>  >> currently not available are not considered, when
> >>>  >>      * evaluating the current chain of {@link
> >>>  >> org.apache.tamaya.spi.PropertySource} instances.
> >>>  >>      * @return true, if a filter is available.
> >>>  >>      */
> >>>  >>     public boolean isAvailable();
> >>>  >>
> >>>  >>     /**
> >>>  >>      * Maps the current properties map to the filtered version
> > of the
> >>>  map.
> >>>  >> The resulting map will be used as the input
> >>>  >>      * for all subsequent handling of PropertySources in the
> >>>  >> PropertySource chain.
> >>>  >>      * @param currentMap the current property map, not null
> >>>  >>      * @return the filtered map, never null.
> >>>  >>      */
> >>>  >>     Map<String,String> map(Map<String,String>
> > currentMap);
> >>>  >>
> >>>  >> }
> >>>  >>
> >>>  >> WDYT?
> >>>  >>
> >>>  >> Anatole
> >>>  >>
> >>>  >>
> >>>  >> 2014-12-29 17:59 GMT+01:00 Anatole Tresch
> > <an...@apache.org>:
> >>>  >>
> >>>  >>> Whereas overrides are simple with this mechanism. I am
> > asking how we
> >>>  want
> >>>  >>> to do other aggregation types, or filtering of values.
> >>>  >>>
> >>>  >>> *On the API side* filtering is always simple, since we
> > have the
> >>>  >>> functional extension points
> > (UnaryOperator<Configuration>), where we
> >>>  can
> >>>  >>> easily filter the whole configuration, and the adapters
> > for filtering
> >>>  >>> single entries on access.
> >>>  >>>
> >>>  >>> But let's assume the following use case:
> >>>  >>>
> >>>  >>> *We have a system low level configuration S, which also
> > contains a few
> >>>  >>> entries, I do not want to see on application config A.
> > Given our
> >>>  priority
> >>>  >>> based assembly this is not possible, since I only can
> > override the
> >>>  values
> >>>  >>> from my parent sources, but never remove them.*
> >>>  >>>
> >>>  >>> Thinking on this the following ideas come to my mind:
> >>>  >>>
> >>>  >>> 1) *add PropertySource filters to my chain. I could use
> > the same
> >>>  >>> priorities. The bad thing is that it is not clear if the
> > filter should
> >>>  be
> >>>  >>> active also, when no source with a similar priority is
> > active and if
> >>>  so if
> >>>  >>> it should be before or after, so this will not work.*
> >>>  >>> *2) I could define a PropertySource to be filtering, so it
> > is able not
> >>>  >>> only to override, but also to remove entries. This
> > "filter" can be
> >>>  >>> activated or deactivated depending on the current context.
> > Given that
> >>>  (if
> >>>  >>> we have set a correct priority value), all works fine.*
> >>>  >>> *3) I could define a separate filter layer which is
> > allowed to filter
> >>>  the
> >>>  >>> entries, before they are published (after all overrides
> > were done).
> >>>  This
> >>>  >>> would probably be transparently be hooked into the
> > "Configuration
> >>>  Layer",
> >>>  >>> which takes the current property sources and creates the
> > final map of
> >>>  >>> properties. Then on the final Map the filter service is
> > operating
> >>>  before
> >>>  >>> any data is handed out to the callers.*
> >>>  >>>
> >>>  >>> I think (3) is the best solution here. Nevertheless we
> > should think on
> >>>  >>> how this can be done the best way and how the SPI here
> > would look like.
> >>>  >>> Also I think, we might have several filters active, e.g.
> > one for
> >>>  logging,
> >>>  >>> one for admins, one which considers the current user's
> > role (aka
> >>>  views), ...
> >>>  >>>
> >>>  >>> So ideas are welcome ;)
> >>>  >>>
> >>>  >>> Best,
> >>>  >>> Anatole
> >>>  >>>
> >>>  >>>
> >>>  >>
> >>>
> >>
> >>
> >>
> >>  --
> >>  *Anatole Tresch*
> >>  Java Engineer & Architect, JSR Spec Lead
> >>  Glärnischweg 10
> >>  CH - 8620 Wetzikon
> >>
> >>  *Switzerland, Europe Zurich, GMT+1*
> >>  *Twitter:  @atsticks*
> >>  *Blogs: **http://javaremarkables.blogspot.ch/
> >>  <http://javaremarkables.blogspot.ch/>*
> >>
> >>  *Google: atsticksMobile  +41-76 344 62 79*
> >
>



-- 
*Anatole Tresch*
Java Engineer & Architect, JSR Spec Lead
Glärnischweg 10
CH - 8620 Wetzikon

*Switzerland, Europe Zurich, GMT+1*
*Twitter:  @atsticks*
*Blogs: **http://javaremarkables.blogspot.ch/
<http://javaremarkables.blogspot.ch/>*

*Google: atsticksMobile  +41-76 344 62 79*

Re: PropertyFiltering

[Mark Struberg <st...@yahoo.de>]

If lazy or eager is a matter of each specific Filter. There is no common answer to that.

The only reason why we have a problem with it is that the the proposed interface handles the whole map. 

If we would instead just apply the filtering chain on single attributes (while they get queried) then all those problems would vanish.

There is btw also a difference between filtering to change values (e.g. decrypt ciphered passwords) and filtering for the purpose of logging.

LieGrue,
strub






> On Thursday, 1 January 2015, 11:25, Romain Manni-Bucau <rm...@gmail.com> wrote:
> > I think we need both but eager initialization should be the default
> (isScannable = true). All other usages (isScannable = false) should be
> lazy and used with caution (main case I have in mind is JNDI and I
> wouldnt rely on it really today).
> 
> What I did in my common impl of conf was to build a Map<String,
> String> then "sources" were able to trigger a refresh event making 
> the
> configuration rebuilding the Map then do a diff and switch if
> different. This implies only eager eval and makes conf management very
> easy but still allowing updates. Only case it doesnt handle is lazy
> eval (JNDI) but this was intended cause harder to
> manage/control/monitor - view would be partial and then wrong if
> allowed.
> 
> 
> 
> Romain Manni-Bucau
> @rmannibucau
> http://www.tomitribe.com
> http://rmannibucau.wordpress.com
> https://github.com/rmannibucau
> 
> 
> 
> 2015-01-01 11:18 GMT+01:00 Anatole Tresch <at...@gmail.com>:
>>  I would say lazyness for evaluation is a must (due to possible environment
>>  dependencies properties returned can be different!), caching of aggregated
>>  configuration on some level comes on top. Or did I get something wrong...?
>> 
>> 
>>  2015-01-01 11:11 GMT+01:00 Romain Manni-Bucau 
> <rm...@gmail.com>:
>> 
>>>  wouldnt it make it harder to implement - ie not user friendly but impl
>>>  friendly only?
>>> 
>>>  + not sure lazyness is useful we need an aggregated config anyway (= 
> eager)
>>> 
>>> 
>>>  Romain Manni-Bucau
>>>  @rmannibucau
>>>  http://www.tomitribe.com
>>>  http://rmannibucau.wordpress.com
>>>  https://github.com/rmannibucau
>>> 
>>> 
>>>  2015-01-01 11:08 GMT+01:00 Anatole Tresch <an...@apache.org>:
>>>  > Hi all
>>>  >
>>>  > another alternative that comes to my mind, would be to model the
>>>  PropertySource
>>>  > more as a Stream<Map.Entry<String,String>> than a 
> provider:
>>>  >
>>>  >    /**
>>>  >      * <p>Evaluate the current properties as Map, based on 
> the current
>>>  > defined properties. The resulting Map may not        * return all 
> items
>>>  > accessible, when the PropertySource is filtering values for the 
> context,
>>>  > where the
>>>  >      * PropertySource is available.</p>
>>>  >      * <p>PropertySources that are not filtering data, 
> should either add
>>>  or
>>>  > override entries, but not remove
>>>  >      * any entries provided by the input map.
>>>  >      *
>>>  >      * @return the a corresponding map, never null.
>>>  >      * //X TODO or should we just do getPropertyKeys()? Think 
> about
>>>  > security (key) vs easier merging (full map)?
>>>  >      */
>>>  >     Stream<Map.Entry<String,String>> getProperties(
>>>  > Stream<Map.Entry<String,String>> 
> currentEntriesStream);
>>>  >
>>>  > This allows us to export all collecting and filtering activities 
> into the
>>>  > Stream API, and also adding entries is simple:
>>>  >
>>>  > Stream stream = Stream.concat(stream1, stream2);
>>>  >
>>>  > Our code gets bare simple and we profit from all the lazy 
> evaluation
>>>  > features....
>>>  >
>>>  > WDYT?
>>>  >
>>>  >
>>>  >
>>>  > 2015-01-01 10:44 GMT+01:00 Anatole Tresch 
> <an...@apache.org>:
>>>  >
>>>  >> Hi all
>>>  >>
>>>  >> I would like to make a *SPI proposal* for PropertyFilter (on 
> API side we
>>>  >> have UnaryOperator<Configuration>). Hereby filters have 
> ordinals and
>>>  >> undergo the same ordering mechanism as PropertySources. 
> Filters get
>>>  >> similarly added to chain of evaluation as PropertySources
>>>  (implementations
>>>  >> must add @Priority -> do we need a programmatic 
> getOrdinal() to have a
>>>  more
>>>  >> concise usae here?), with one exceptional rule:
>>>  >> If a filter accidentally has the same ordinal as some 
> PropertySources in
>>>  >> the chain, filters are always executed AFTER PropertySources
>>>  >>
>>>  >> So here is the interface:.
>>>  >>
>>>  >> /**
>>>  >>  * <p>Interface for filtering the current map of 
> properties during the
>>>  >> evaluation of the chain of PropertySources.
>>>  >>  * Filters can be registered using the {@link
>>>  >> org.apache.tamaya.spi.ServiceContext} or added 
> programmatically
>>>  >>  * by calling {@link
>>>  >> 
> org.apache.tamaya.spi.ConfigurationContext#addPropertyFilter(int,
>>>  >> PropertyFilter)}. The ordinal
>>>  >>  * hereby is defined by the corresponding {@code 
> @Priority}
>>>  annotation.</p>
>>>  >>  * <p>Filters </p>
>>>  >>  */
>>>  >> public interface PropertyFilter<T>{
>>>  >>
>>>  >>     /**
>>>  >>      * Method to check, if a filter is available. Filters that 
> are
>>>  >> currently not available are not considered, when
>>>  >>      * evaluating the current chain of {@link
>>>  >> org.apache.tamaya.spi.PropertySource} instances.
>>>  >>      * @return true, if a filter is available.
>>>  >>      */
>>>  >>     public boolean isAvailable();
>>>  >>
>>>  >>     /**
>>>  >>      * Maps the current properties map to the filtered version 
> of the
>>>  map.
>>>  >> The resulting map will be used as the input
>>>  >>      * for all subsequent handling of PropertySources in the
>>>  >> PropertySource chain.
>>>  >>      * @param currentMap the current property map, not null
>>>  >>      * @return the filtered map, never null.
>>>  >>      */
>>>  >>     Map<String,String> map(Map<String,String> 
> currentMap);
>>>  >>
>>>  >> }
>>>  >>
>>>  >> WDYT?
>>>  >>
>>>  >> Anatole
>>>  >>
>>>  >>
>>>  >> 2014-12-29 17:59 GMT+01:00 Anatole Tresch 
> <an...@apache.org>:
>>>  >>
>>>  >>> Whereas overrides are simple with this mechanism. I am 
> asking how we
>>>  want
>>>  >>> to do other aggregation types, or filtering of values.
>>>  >>>
>>>  >>> *On the API side* filtering is always simple, since we 
> have the
>>>  >>> functional extension points 
> (UnaryOperator<Configuration>), where we
>>>  can
>>>  >>> easily filter the whole configuration, and the adapters 
> for filtering
>>>  >>> single entries on access.
>>>  >>>
>>>  >>> But let's assume the following use case:
>>>  >>>
>>>  >>> *We have a system low level configuration S, which also 
> contains a few
>>>  >>> entries, I do not want to see on application config A. 
> Given our
>>>  priority
>>>  >>> based assembly this is not possible, since I only can 
> override the
>>>  values
>>>  >>> from my parent sources, but never remove them.*
>>>  >>>
>>>  >>> Thinking on this the following ideas come to my mind:
>>>  >>>
>>>  >>> 1) *add PropertySource filters to my chain. I could use 
> the same
>>>  >>> priorities. The bad thing is that it is not clear if the 
> filter should
>>>  be
>>>  >>> active also, when no source with a similar priority is 
> active and if
>>>  so if
>>>  >>> it should be before or after, so this will not work.*
>>>  >>> *2) I could define a PropertySource to be filtering, so it 
> is able not
>>>  >>> only to override, but also to remove entries. This 
> "filter" can be
>>>  >>> activated or deactivated depending on the current context. 
> Given that
>>>  (if
>>>  >>> we have set a correct priority value), all works fine.*
>>>  >>> *3) I could define a separate filter layer which is 
> allowed to filter
>>>  the
>>>  >>> entries, before they are published (after all overrides 
> were done).
>>>  This
>>>  >>> would probably be transparently be hooked into the 
> "Configuration
>>>  Layer",
>>>  >>> which takes the current property sources and creates the 
> final map of
>>>  >>> properties. Then on the final Map the filter service is 
> operating
>>>  before
>>>  >>> any data is handed out to the callers.*
>>>  >>>
>>>  >>> I think (3) is the best solution here. Nevertheless we 
> should think on
>>>  >>> how this can be done the best way and how the SPI here 
> would look like.
>>>  >>> Also I think, we might have several filters active, e.g. 
> one for
>>>  logging,
>>>  >>> one for admins, one which considers the current user's 
> role (aka
>>>  views), ...
>>>  >>>
>>>  >>> So ideas are welcome ;)
>>>  >>>
>>>  >>> Best,
>>>  >>> Anatole
>>>  >>>
>>>  >>>
>>>  >>
>>> 
>> 
>> 
>> 
>>  --
>>  *Anatole Tresch*
>>  Java Engineer & Architect, JSR Spec Lead
>>  Glärnischweg 10
>>  CH - 8620 Wetzikon
>> 
>>  *Switzerland, Europe Zurich, GMT+1*
>>  *Twitter:  @atsticks*
>>  *Blogs: **http://javaremarkables.blogspot.ch/
>>  <http://javaremarkables.blogspot.ch/>*
>> 
>>  *Google: atsticksMobile  +41-76 344 62 79*
>

Re: PropertyFiltering

[Romain Manni-Bucau <rm...@gmail.com>]

I think we need both but eager initialization should be the default
(isScannable = true). All other usages (isScannable = false) should be
lazy and used with caution (main case I have in mind is JNDI and I
wouldnt rely on it really today).

What I did in my common impl of conf was to build a Map<String,
String> then "sources" were able to trigger a refresh event making the
configuration rebuilding the Map then do a diff and switch if
different. This implies only eager eval and makes conf management very
easy but still allowing updates. Only case it doesnt handle is lazy
eval (JNDI) but this was intended cause harder to
manage/control/monitor - view would be partial and then wrong if
allowed.



Romain Manni-Bucau
@rmannibucau
http://www.tomitribe.com
http://rmannibucau.wordpress.com
https://github.com/rmannibucau


2015-01-01 11:18 GMT+01:00 Anatole Tresch <at...@gmail.com>:
> I would say lazyness for evaluation is a must (due to possible environment
> dependencies properties returned can be different!), caching of aggregated
> configuration on some level comes on top. Or did I get something wrong...?
>
>
> 2015-01-01 11:11 GMT+01:00 Romain Manni-Bucau <rm...@gmail.com>:
>
>> wouldnt it make it harder to implement - ie not user friendly but impl
>> friendly only?
>>
>> + not sure lazyness is useful we need an aggregated config anyway (= eager)
>>
>>
>> Romain Manni-Bucau
>> @rmannibucau
>> http://www.tomitribe.com
>> http://rmannibucau.wordpress.com
>> https://github.com/rmannibucau
>>
>>
>> 2015-01-01 11:08 GMT+01:00 Anatole Tresch <an...@apache.org>:
>> > Hi all
>> >
>> > another alternative that comes to my mind, would be to model the
>> PropertySource
>> > more as a Stream<Map.Entry<String,String>> than a provider:
>> >
>> >    /**
>> >      * <p>Evaluate the current properties as Map, based on the current
>> > defined properties. The resulting Map may not        * return all items
>> > accessible, when the PropertySource is filtering values for the context,
>> > where the
>> >      * PropertySource is available.</p>
>> >      * <p>PropertySources that are not filtering data, should either add
>> or
>> > override entries, but not remove
>> >      * any entries provided by the input map.
>> >      *
>> >      * @return the a corresponding map, never null.
>> >      * //X TODO or should we just do getPropertyKeys()? Think about
>> > security (key) vs easier merging (full map)?
>> >      */
>> >     Stream<Map.Entry<String,String>> getProperties(
>> > Stream<Map.Entry<String,String>> currentEntriesStream);
>> >
>> > This allows us to export all collecting and filtering activities into the
>> > Stream API, and also adding entries is simple:
>> >
>> > Stream stream = Stream.concat(stream1, stream2);
>> >
>> > Our code gets bare simple and we profit from all the lazy evaluation
>> > features....
>> >
>> > WDYT?
>> >
>> >
>> >
>> > 2015-01-01 10:44 GMT+01:00 Anatole Tresch <an...@apache.org>:
>> >
>> >> Hi all
>> >>
>> >> I would like to make a *SPI proposal* for PropertyFilter (on API side we
>> >> have UnaryOperator<Configuration>). Hereby filters have ordinals and
>> >> undergo the same ordering mechanism as PropertySources. Filters get
>> >> similarly added to chain of evaluation as PropertySources
>> (implementations
>> >> must add @Priority -> do we need a programmatic getOrdinal() to have a
>> more
>> >> concise usae here?), with one exceptional rule:
>> >> If a filter accidentally has the same ordinal as some PropertySources in
>> >> the chain, filters are always executed AFTER PropertySources
>> >>
>> >> So here is the interface:.
>> >>
>> >> /**
>> >>  * <p>Interface for filtering the current map of properties during the
>> >> evaluation of the chain of PropertySources.
>> >>  * Filters can be registered using the {@link
>> >> org.apache.tamaya.spi.ServiceContext} or added programmatically
>> >>  * by calling {@link
>> >> org.apache.tamaya.spi.ConfigurationContext#addPropertyFilter(int,
>> >> PropertyFilter)}. The ordinal
>> >>  * hereby is defined by the corresponding {@code @Priority}
>> annotation.</p>
>> >>  * <p>Filters </p>
>> >>  */
>> >> public interface PropertyFilter<T>{
>> >>
>> >>     /**
>> >>      * Method to check, if a filter is available. Filters that are
>> >> currently not available are not considered, when
>> >>      * evaluating the current chain of {@link
>> >> org.apache.tamaya.spi.PropertySource} instances.
>> >>      * @return true, if a filter is available.
>> >>      */
>> >>     public boolean isAvailable();
>> >>
>> >>     /**
>> >>      * Maps the current properties map to the filtered version of the
>> map.
>> >> The resulting map will be used as the input
>> >>      * for all subsequent handling of PropertySources in the
>> >> PropertySource chain.
>> >>      * @param currentMap the current property map, not null
>> >>      * @return the filtered map, never null.
>> >>      */
>> >>     Map<String,String> map(Map<String,String> currentMap);
>> >>
>> >> }
>> >>
>> >> WDYT?
>> >>
>> >> Anatole
>> >>
>> >>
>> >> 2014-12-29 17:59 GMT+01:00 Anatole Tresch <an...@apache.org>:
>> >>
>> >>> Whereas overrides are simple with this mechanism. I am asking how we
>> want
>> >>> to do other aggregation types, or filtering of values.
>> >>>
>> >>> *On the API side* filtering is always simple, since we have the
>> >>> functional extension points (UnaryOperator<Configuration>), where we
>> can
>> >>> easily filter the whole configuration, and the adapters for filtering
>> >>> single entries on access.
>> >>>
>> >>> But let's assume the following use case:
>> >>>
>> >>> *We have a system low level configuration S, which also contains a few
>> >>> entries, I do not want to see on application config A. Given our
>> priority
>> >>> based assembly this is not possible, since I only can override the
>> values
>> >>> from my parent sources, but never remove them.*
>> >>>
>> >>> Thinking on this the following ideas come to my mind:
>> >>>
>> >>> 1) *add PropertySource filters to my chain. I could use the same
>> >>> priorities. The bad thing is that it is not clear if the filter should
>> be
>> >>> active also, when no source with a similar priority is active and if
>> so if
>> >>> it should be before or after, so this will not work.*
>> >>> *2) I could define a PropertySource to be filtering, so it is able not
>> >>> only to override, but also to remove entries. This "filter" can be
>> >>> activated or deactivated depending on the current context. Given that
>> (if
>> >>> we have set a correct priority value), all works fine.*
>> >>> *3) I could define a separate filter layer which is allowed to filter
>> the
>> >>> entries, before they are published (after all overrides were done).
>> This
>> >>> would probably be transparently be hooked into the "Configuration
>> Layer",
>> >>> which takes the current property sources and creates the final map of
>> >>> properties. Then on the final Map the filter service is operating
>> before
>> >>> any data is handed out to the callers.*
>> >>>
>> >>> I think (3) is the best solution here. Nevertheless we should think on
>> >>> how this can be done the best way and how the SPI here would look like.
>> >>> Also I think, we might have several filters active, e.g. one for
>> logging,
>> >>> one for admins, one which considers the current user's role (aka
>> views), ...
>> >>>
>> >>> So ideas are welcome ;)
>> >>>
>> >>> Best,
>> >>> Anatole
>> >>>
>> >>>
>> >>
>>
>
>
>
> --
> *Anatole Tresch*
> Java Engineer & Architect, JSR Spec Lead
> Glärnischweg 10
> CH - 8620 Wetzikon
>
> *Switzerland, Europe Zurich, GMT+1*
> *Twitter:  @atsticks*
> *Blogs: **http://javaremarkables.blogspot.ch/
> <http://javaremarkables.blogspot.ch/>*
>
> *Google: atsticksMobile  +41-76 344 62 79*

Re: PropertyFiltering

[Anatole Tresch <at...@gmail.com>]

I would say lazyness for evaluation is a must (due to possible environment
dependencies properties returned can be different!), caching of aggregated
configuration on some level comes on top. Or did I get something wrong...?


2015-01-01 11:11 GMT+01:00 Romain Manni-Bucau <rm...@gmail.com>:

> wouldnt it make it harder to implement - ie not user friendly but impl
> friendly only?
>
> + not sure lazyness is useful we need an aggregated config anyway (= eager)
>
>
> Romain Manni-Bucau
> @rmannibucau
> http://www.tomitribe.com
> http://rmannibucau.wordpress.com
> https://github.com/rmannibucau
>
>
> 2015-01-01 11:08 GMT+01:00 Anatole Tresch <an...@apache.org>:
> > Hi all
> >
> > another alternative that comes to my mind, would be to model the
> PropertySource
> > more as a Stream<Map.Entry<String,String>> than a provider:
> >
> >    /**
> >      * <p>Evaluate the current properties as Map, based on the current
> > defined properties. The resulting Map may not        * return all items
> > accessible, when the PropertySource is filtering values for the context,
> > where the
> >      * PropertySource is available.</p>
> >      * <p>PropertySources that are not filtering data, should either add
> or
> > override entries, but not remove
> >      * any entries provided by the input map.
> >      *
> >      * @return the a corresponding map, never null.
> >      * //X TODO or should we just do getPropertyKeys()? Think about
> > security (key) vs easier merging (full map)?
> >      */
> >     Stream<Map.Entry<String,String>> getProperties(
> > Stream<Map.Entry<String,String>> currentEntriesStream);
> >
> > This allows us to export all collecting and filtering activities into the
> > Stream API, and also adding entries is simple:
> >
> > Stream stream = Stream.concat(stream1, stream2);
> >
> > Our code gets bare simple and we profit from all the lazy evaluation
> > features....
> >
> > WDYT?
> >
> >
> >
> > 2015-01-01 10:44 GMT+01:00 Anatole Tresch <an...@apache.org>:
> >
> >> Hi all
> >>
> >> I would like to make a *SPI proposal* for PropertyFilter (on API side we
> >> have UnaryOperator<Configuration>). Hereby filters have ordinals and
> >> undergo the same ordering mechanism as PropertySources. Filters get
> >> similarly added to chain of evaluation as PropertySources
> (implementations
> >> must add @Priority -> do we need a programmatic getOrdinal() to have a
> more
> >> concise usae here?), with one exceptional rule:
> >> If a filter accidentally has the same ordinal as some PropertySources in
> >> the chain, filters are always executed AFTER PropertySources
> >>
> >> So here is the interface:.
> >>
> >> /**
> >>  * <p>Interface for filtering the current map of properties during the
> >> evaluation of the chain of PropertySources.
> >>  * Filters can be registered using the {@link
> >> org.apache.tamaya.spi.ServiceContext} or added programmatically
> >>  * by calling {@link
> >> org.apache.tamaya.spi.ConfigurationContext#addPropertyFilter(int,
> >> PropertyFilter)}. The ordinal
> >>  * hereby is defined by the corresponding {@code @Priority}
> annotation.</p>
> >>  * <p>Filters </p>
> >>  */
> >> public interface PropertyFilter<T>{
> >>
> >>     /**
> >>      * Method to check, if a filter is available. Filters that are
> >> currently not available are not considered, when
> >>      * evaluating the current chain of {@link
> >> org.apache.tamaya.spi.PropertySource} instances.
> >>      * @return true, if a filter is available.
> >>      */
> >>     public boolean isAvailable();
> >>
> >>     /**
> >>      * Maps the current properties map to the filtered version of the
> map.
> >> The resulting map will be used as the input
> >>      * for all subsequent handling of PropertySources in the
> >> PropertySource chain.
> >>      * @param currentMap the current property map, not null
> >>      * @return the filtered map, never null.
> >>      */
> >>     Map<String,String> map(Map<String,String> currentMap);
> >>
> >> }
> >>
> >> WDYT?
> >>
> >> Anatole
> >>
> >>
> >> 2014-12-29 17:59 GMT+01:00 Anatole Tresch <an...@apache.org>:
> >>
> >>> Whereas overrides are simple with this mechanism. I am asking how we
> want
> >>> to do other aggregation types, or filtering of values.
> >>>
> >>> *On the API side* filtering is always simple, since we have the
> >>> functional extension points (UnaryOperator<Configuration>), where we
> can
> >>> easily filter the whole configuration, and the adapters for filtering
> >>> single entries on access.
> >>>
> >>> But let's assume the following use case:
> >>>
> >>> *We have a system low level configuration S, which also contains a few
> >>> entries, I do not want to see on application config A. Given our
> priority
> >>> based assembly this is not possible, since I only can override the
> values
> >>> from my parent sources, but never remove them.*
> >>>
> >>> Thinking on this the following ideas come to my mind:
> >>>
> >>> 1) *add PropertySource filters to my chain. I could use the same
> >>> priorities. The bad thing is that it is not clear if the filter should
> be
> >>> active also, when no source with a similar priority is active and if
> so if
> >>> it should be before or after, so this will not work.*
> >>> *2) I could define a PropertySource to be filtering, so it is able not
> >>> only to override, but also to remove entries. This "filter" can be
> >>> activated or deactivated depending on the current context. Given that
> (if
> >>> we have set a correct priority value), all works fine.*
> >>> *3) I could define a separate filter layer which is allowed to filter
> the
> >>> entries, before they are published (after all overrides were done).
> This
> >>> would probably be transparently be hooked into the "Configuration
> Layer",
> >>> which takes the current property sources and creates the final map of
> >>> properties. Then on the final Map the filter service is operating
> before
> >>> any data is handed out to the callers.*
> >>>
> >>> I think (3) is the best solution here. Nevertheless we should think on
> >>> how this can be done the best way and how the SPI here would look like.
> >>> Also I think, we might have several filters active, e.g. one for
> logging,
> >>> one for admins, one which considers the current user's role (aka
> views), ...
> >>>
> >>> So ideas are welcome ;)
> >>>
> >>> Best,
> >>> Anatole
> >>>
> >>>
> >>
>



-- 
*Anatole Tresch*
Java Engineer & Architect, JSR Spec Lead
Glärnischweg 10
CH - 8620 Wetzikon

*Switzerland, Europe Zurich, GMT+1*
*Twitter:  @atsticks*
*Blogs: **http://javaremarkables.blogspot.ch/
<http://javaremarkables.blogspot.ch/>*

*Google: atsticksMobile  +41-76 344 62 79*

Re: PropertyFiltering

[Romain Manni-Bucau <rm...@gmail.com>]

wouldnt it make it harder to implement - ie not user friendly but impl
friendly only?

+ not sure lazyness is useful we need an aggregated config anyway (= eager)


Romain Manni-Bucau
@rmannibucau
http://www.tomitribe.com
http://rmannibucau.wordpress.com
https://github.com/rmannibucau


2015-01-01 11:08 GMT+01:00 Anatole Tresch <an...@apache.org>:
> Hi all
>
> another alternative that comes to my mind, would be to model the PropertySource
> more as a Stream<Map.Entry<String,String>> than a provider:
>
>    /**
>      * <p>Evaluate the current properties as Map, based on the current
> defined properties. The resulting Map may not        * return all items
> accessible, when the PropertySource is filtering values for the context,
> where the
>      * PropertySource is available.</p>
>      * <p>PropertySources that are not filtering data, should either add or
> override entries, but not remove
>      * any entries provided by the input map.
>      *
>      * @return the a corresponding map, never null.
>      * //X TODO or should we just do getPropertyKeys()? Think about
> security (key) vs easier merging (full map)?
>      */
>     Stream<Map.Entry<String,String>> getProperties(
> Stream<Map.Entry<String,String>> currentEntriesStream);
>
> This allows us to export all collecting and filtering activities into the
> Stream API, and also adding entries is simple:
>
> Stream stream = Stream.concat(stream1, stream2);
>
> Our code gets bare simple and we profit from all the lazy evaluation
> features....
>
> WDYT?
>
>
>
> 2015-01-01 10:44 GMT+01:00 Anatole Tresch <an...@apache.org>:
>
>> Hi all
>>
>> I would like to make a *SPI proposal* for PropertyFilter (on API side we
>> have UnaryOperator<Configuration>). Hereby filters have ordinals and
>> undergo the same ordering mechanism as PropertySources. Filters get
>> similarly added to chain of evaluation as PropertySources (implementations
>> must add @Priority -> do we need a programmatic getOrdinal() to have a more
>> concise usae here?), with one exceptional rule:
>> If a filter accidentally has the same ordinal as some PropertySources in
>> the chain, filters are always executed AFTER PropertySources
>>
>> So here is the interface:.
>>
>> /**
>>  * <p>Interface for filtering the current map of properties during the
>> evaluation of the chain of PropertySources.
>>  * Filters can be registered using the {@link
>> org.apache.tamaya.spi.ServiceContext} or added programmatically
>>  * by calling {@link
>> org.apache.tamaya.spi.ConfigurationContext#addPropertyFilter(int,
>> PropertyFilter)}. The ordinal
>>  * hereby is defined by the corresponding {@code @Priority} annotation.</p>
>>  * <p>Filters </p>
>>  */
>> public interface PropertyFilter<T>{
>>
>>     /**
>>      * Method to check, if a filter is available. Filters that are
>> currently not available are not considered, when
>>      * evaluating the current chain of {@link
>> org.apache.tamaya.spi.PropertySource} instances.
>>      * @return true, if a filter is available.
>>      */
>>     public boolean isAvailable();
>>
>>     /**
>>      * Maps the current properties map to the filtered version of the map.
>> The resulting map will be used as the input
>>      * for all subsequent handling of PropertySources in the
>> PropertySource chain.
>>      * @param currentMap the current property map, not null
>>      * @return the filtered map, never null.
>>      */
>>     Map<String,String> map(Map<String,String> currentMap);
>>
>> }
>>
>> WDYT?
>>
>> Anatole
>>
>>
>> 2014-12-29 17:59 GMT+01:00 Anatole Tresch <an...@apache.org>:
>>
>>> Whereas overrides are simple with this mechanism. I am asking how we want
>>> to do other aggregation types, or filtering of values.
>>>
>>> *On the API side* filtering is always simple, since we have the
>>> functional extension points (UnaryOperator<Configuration>), where we can
>>> easily filter the whole configuration, and the adapters for filtering
>>> single entries on access.
>>>
>>> But let's assume the following use case:
>>>
>>> *We have a system low level configuration S, which also contains a few
>>> entries, I do not want to see on application config A. Given our priority
>>> based assembly this is not possible, since I only can override the values
>>> from my parent sources, but never remove them.*
>>>
>>> Thinking on this the following ideas come to my mind:
>>>
>>> 1) *add PropertySource filters to my chain. I could use the same
>>> priorities. The bad thing is that it is not clear if the filter should be
>>> active also, when no source with a similar priority is active and if so if
>>> it should be before or after, so this will not work.*
>>> *2) I could define a PropertySource to be filtering, so it is able not
>>> only to override, but also to remove entries. This "filter" can be
>>> activated or deactivated depending on the current context. Given that (if
>>> we have set a correct priority value), all works fine.*
>>> *3) I could define a separate filter layer which is allowed to filter the
>>> entries, before they are published (after all overrides were done). This
>>> would probably be transparently be hooked into the "Configuration Layer",
>>> which takes the current property sources and creates the final map of
>>> properties. Then on the final Map the filter service is operating before
>>> any data is handed out to the callers.*
>>>
>>> I think (3) is the best solution here. Nevertheless we should think on
>>> how this can be done the best way and how the SPI here would look like.
>>> Also I think, we might have several filters active, e.g. one for logging,
>>> one for admins, one which considers the current user's role (aka views), ...
>>>
>>> So ideas are welcome ;)
>>>
>>> Best,
>>> Anatole
>>>
>>>
>>

Re: PropertyFiltering

[Anatole Tresch <an...@apache.org>]

Hi all

another alternative that comes to my mind, would be to model the PropertySource
more as a Stream<Map.Entry<String,String>> than a provider:

   /**
     * <p>Evaluate the current properties as Map, based on the current
defined properties. The resulting Map may not        * return all items
accessible, when the PropertySource is filtering values for the context,
where the
     * PropertySource is available.</p>
     * <p>PropertySources that are not filtering data, should either add or
override entries, but not remove
     * any entries provided by the input map.
     *
     * @return the a corresponding map, never null.
     * //X TODO or should we just do getPropertyKeys()? Think about
security (key) vs easier merging (full map)?
     */
    Stream<Map.Entry<String,String>> getProperties(
Stream<Map.Entry<String,String>> currentEntriesStream);

This allows us to export all collecting and filtering activities into the
Stream API, and also adding entries is simple:

Stream stream = Stream.concat(stream1, stream2);

​Our code gets bare simple and we profit from all the lazy evaluation
features....

WDYT?
 ​


2015-01-01 10:44 GMT+01:00 Anatole Tresch <an...@apache.org>:

> Hi all
>
> I would like to make a *SPI proposal* for PropertyFilter (on API side we
> have UnaryOperator<Configuration>). Hereby filters have ordinals and
> undergo the same ordering mechanism as PropertySources. Filters get
> similarly added to chain of evaluation as PropertySources (implementations
> must add @Priority -> do we need a programmatic getOrdinal() to have a more
> concise usae here?), with one exceptional rule:
> If a filter accidentally has the same ordinal as some PropertySources in
> the chain, filters are always executed AFTER PropertySources
>
> So here is the interface:.
>
> /**
>  * <p>Interface for filtering the current map of properties during the
> evaluation of the chain of PropertySources.
>  * Filters can be registered using the {@link
> org.apache.tamaya.spi.ServiceContext} or added programmatically
>  * by calling {@link
> org.apache.tamaya.spi.ConfigurationContext#addPropertyFilter(int,
> PropertyFilter)}. The ordinal
>  * hereby is defined by the corresponding {@code @Priority} annotation.</p>
>  * <p>Filters </p>
>  */
> public interface PropertyFilter<T>{
>
>     /**
>      * Method to check, if a filter is available. Filters that are
> currently not available are not considered, when
>      * evaluating the current chain of {@link
> org.apache.tamaya.spi.PropertySource} instances.
>      * @return true, if a filter is available.
>      */
>     public boolean isAvailable();
>
>     /**
>      * Maps the current properties map to the filtered version of the map.
> The resulting map will be used as the input
>      * for all subsequent handling of PropertySources in the
> PropertySource chain.
>      * @param currentMap the current property map, not null
>      * @return the filtered map, never null.
>      */
>     Map<String,String> map(Map<String,String> currentMap);
>
> }
>
> WDYT?
>
> Anatole
>
>
> 2014-12-29 17:59 GMT+01:00 Anatole Tresch <an...@apache.org>:
>
>> Whereas overrides are simple with this mechanism. I am asking how we want
>> to do other aggregation types, or filtering of values.
>>
>> *On the API side* filtering is always simple, since we have the
>> functional extension points (UnaryOperator<Configuration>), where we can
>> easily filter the whole configuration, and the adapters for filtering
>> single entries on access.
>>
>> But let's assume the following use case:
>>
>> *We have a system low level configuration S, which also contains a few
>> entries, I do not want to see on application config A. Given our priority
>> based assembly this is not possible, since I only can override the values
>> from my parent sources, but never remove them.*
>>
>> Thinking on this the following ideas come to my mind:
>>
>> 1) *add PropertySource filters to my chain. I could use the same
>> priorities. The bad thing is that it is not clear if the filter should be
>> active also, when no source with a similar priority is active and if so if
>> it should be before or after, so this will not work.*
>> *2) I could define a PropertySource to be filtering, so it is able not
>> only to override, but also to remove entries. This "filter" can be
>> activated or deactivated depending on the current context. Given that (if
>> we have set a correct priority value), all works fine.*
>> *3) I could define a separate filter layer which is allowed to filter the
>> entries, before they are published (after all overrides were done). This
>> would probably be transparently be hooked into the "Configuration Layer",
>> which takes the current property sources and creates the final map of
>> properties. Then on the final Map the filter service is operating before
>> any data is handed out to the callers.*
>>
>> I think (3) is the best solution here. Nevertheless we should think on
>> how this can be done the best way and how the SPI here would look like.
>> Also I think, we might have several filters active, e.g. one for logging,
>> one for admins, one which considers the current user's role (aka views), ...
>>
>> So ideas are welcome ;)
>>
>> Best,
>> Anatole
>>
>>
>