aether url handler problems and proposed solutions

[David Jencks <da...@yahoo.com>]

At the risk of causing total confusion, I'm going to discuss more than one issue in this email.  Please consider looking at the whole message, not just the first problem.

1. Configuration source (PAXURL-147, KARAF-910).  Karaf uses the pax mvn (aether) url handler during initial startup, and tries to supply a config admin based configuration for it.  Currently this is stymied by the behavior of using framework properties initially, then replacing them with config admin configuration when available.  Here are some requirements:

-  Karaf needs a solution whereby the url handler is not registered until the config admin configuration is in place.
- The url handler needs to be usable with no configuration at all
- The url handler needs to be usable with framework property configuration

proposed solution:
1.a. add a flag org.ops4j.pax.url.mvn.requireConfigAdminConfig that can be configured in framework properties.  When set in a configuration source, the configuration source will be treated as not valid and will not result in the url handler being registered.
1.b make the config admin based configuration NOT use the framework property based configuration as fallback.  This means that the config admin based configuration source will not see the flag above if it's set only in the framework properties.

I think that 1.b is a good change in any case.  If you want to use config admin based configuration, use it!

With a couple simple changes in karaf feature service to wait for the url handler, this seems to solve the problems we've been having with karaf as far as using an unconfigured url handler.

2. Problems with repo management in the aether url handler.

2.1 (PAXURL-149) Mirrored repos are not fully mirrored: they are still consulted directly.  This really defeats the purpose of mirroring.  In more detail, if a repo is mirrored, the mirror is added to the beginning of the list of remote repos, but the mirrored repo is left in the list. So for instance if you have configured your local nexus repo to block downloading of some urls, the presence of the original repo after the mirror will mean that the urls can be downloaded directly anyway.

Solution:  remove the mirrored repos from the list of remote repos when adding the mirror.

2.2  (PAXURL-150) Local repository handling is too inflexible.  Aether can deal with only one local repo, as the place stuff is downloaded to.  In karaf, we have a "system" repo inside the karaf directory structure that should be used as the download location.  This is required because it's possible to get a bundle in the system repo that you want to install using a mvn: url but you do NOT want to get into the ~/.m2/repository local maven repo.  If karaf uses the ~/.m2/repository as the local repo and adds the karaf system as an additional remote repo, aether will "download" the bundle into the ~/.m2/repository repo.  Therefore the karaf system repo should be configured as the local repository and the ~/.m2/repository local maven repository should be configured as an additional remote repo.  However the location of this repo is not known in advance since it can be configured in settings.xml.

Solution: add a flag org.ops4j.pax.url.mvn.defaultLocalRepoAsRemote that when set will find the actual location of the maven local repo from settings.xml and add it to the list of remote repos.
Possible alternate solution: if org.ops4j.pax.url.mvn.localRepository is set and differs from the maven local repo, add the maven local repo as a remote repo.  This uses less configuration but is less flexible.



Since these proposals change the behavior of the url handler I would like to discuss them before committing.  On the other hand KARAF-910 has been a serious problem in karaf for a long time so I would like to commit the fixes soon.

Many thanks
david jencks

Re: aether url handler problems and proposed solutions

[Andreas Pieber <an...@gmail.com>]

Hey David,

I'm also +1 (with 1.b and 2.2.a) to all of those changes. I would consider
1.b and 2.1 rather as a bug-fix and 2.2(.a) is an optional feature (from a
user point of view); so no problem here.

Kind regards,
Andreas

On Tue, Jan 10, 2012 at 00:13, David Jencks <da...@yahoo.com> wrote:

> At the risk of causing total confusion, I'm going to discuss more than one
> issue in this email.  Please consider looking at the whole message, not
> just the first problem.
>
> 1. Configuration source (PAXURL-147, KARAF-910).  Karaf uses the pax mvn
> (aether) url handler during initial startup, and tries to supply a config
> admin based configuration for it.  Currently this is stymied by the
> behavior of using framework properties initially, then replacing them with
> config admin configuration when available.  Here are some requirements:
>
> -  Karaf needs a solution whereby the url handler is not registered until
> the config admin configuration is in place.
> - The url handler needs to be usable with no configuration at all
> - The url handler needs to be usable with framework property configuration
>
> proposed solution:
> 1.a. add a flag org.ops4j.pax.url.mvn.requireConfigAdminConfig that can be
> configured in framework properties.  When set in a configuration source,
> the configuration source will be treated as not valid and will not result
> in the url handler being registered.
> 1.b make the config admin based configuration NOT use the framework
> property based configuration as fallback.  This means that the config admin
> based configuration source will not see the flag above if it's set only in
> the framework properties.
>
> I think that 1.b is a good change in any case.  If you want to use config
> admin based configuration, use it!
>
> With a couple simple changes in karaf feature service to wait for the url
> handler, this seems to solve the problems we've been having with karaf as
> far as using an unconfigured url handler.
>
> 2. Problems with repo management in the aether url handler.
>
> 2.1 (PAXURL-149) Mirrored repos are not fully mirrored: they are still
> consulted directly.  This really defeats the purpose of mirroring.  In more
> detail, if a repo is mirrored, the mirror is added to the beginning of the
> list of remote repos, but the mirrored repo is left in the list. So for
> instance if you have configured your local nexus repo to block downloading
> of some urls, the presence of the original repo after the mirror will mean
> that the urls can be downloaded directly anyway.
>
> Solution:  remove the mirrored repos from the list of remote repos when
> adding the mirror.
>
> 2.2  (PAXURL-150) Local repository handling is too inflexible.  Aether can
> deal with only one local repo, as the place stuff is downloaded to.  In
> karaf, we have a "system" repo inside the karaf directory structure that
> should be used as the download location.  This is required because it's
> possible to get a bundle in the system repo that you want to install using
> a mvn: url but you do NOT want to get into the ~/.m2/repository local maven
> repo.  If karaf uses the ~/.m2/repository as the local repo and adds the
> karaf system as an additional remote repo, aether will "download" the
> bundle into the ~/.m2/repository repo.  Therefore the karaf system repo
> should be configured as the local repository and the ~/.m2/repository local
> maven repository should be configured as an additional remote repo.
>  However the location of this repo is not known in advance since it can be
> configured in settings.xml.
>
> Solution: add a flag org.ops4j.pax.url.mvn.defaultLocalRepoAsRemote that
> when set will find the actual location of the maven local repo from
> settings.xml and add it to the list of remote repos.
> Possible alternate solution: if org.ops4j.pax.url.mvn.localRepository is
> set and differs from the maven local repo, add the maven local repo as a
> remote repo.  This uses less configuration but is less flexible.
>
>
>
> Since these proposals change the behavior of the url handler I would like
> to discuss them before committing.  On the other hand KARAF-910 has been a
> serious problem in karaf for a long time so I would like to commit the
> fixes soon.
>
> Many thanks
> david jencks
>
>