You are viewing a plain text version of this content. The canonical link for it is here.
Posted to general@incubator.apache.org by Gurkan Erdogdu <cg...@gmail.com> on 2010/03/08 16:50:11 UTC
Third party Maven Repository Usage
Hi;
Is there any rule/policy for using third-party maven repositories in our
project poms? Recently, we have required to list some repositories in
settings.xml (for example: jboss) to our codebase built correctly. But when
Apache-Hudson runs to built daily, it throws errors because of not finding
required repositories. Is it OK to list those repositories in our poms?
I have just found
http://maven.apache.org/guides/mini/guide-central-repository-upload.htmldocumentation
FAQ and common mistakes section.
Thanks;
--Gurkan
Re: Third party Maven Repository Usage
Posted by Ate Douma <at...@douma.nu>.
Thanks for the reply Brian, this is much relief.
However in your initial response your wording clearly indicated the new policy already being enforced, up to:
"[...] your artifacts will end up blocked."
A little less restrictive description could have prevented this misunderstanding...
I think that, with the goals you have, it would be good to already "announce" this more prominently, especially within the ASF.
I suspect others will raise questions similar to mine and maybe even more.
Upfront and public awareness of important changes things like these is important to build up support for it IMO.
Doing it 1-on-1 with individual release managers/projects seems both very time consuming and "hiding" it from the larger community.
I don't know who "owns" Maven Central (Sonatype?) but as the "default" repository build into Apache Maven I would think at least the ASF
community would have to be informed proper and be allowed to discuss what/how/when concerning such policy changes.
Never mind, we're cool again for now.
I'll ping you again when we are ready for the rsync of our "legacy" bugfix release.
Kind regards,
Ate
On 03/25/2010 05:49 AM, Brian Fox wrote:
>>> Interesting. That's news to me... You have a pointer to more information?
>>
>
> As it turns out, almost all references to external repositories in
> poms are junk or turn out to be junk after a bit of time. See here for
> some examples:
> http://www.sonatype.com/people/2010/03/why-external-repos-are-being-phased-out-of-central/
>
>
>> * Unclear from the documentation is if this restriction on external
>> repositories is limited to only the repository definitions in a pom, or if
>> it is (or will be) extended to dependency resolving as well.
>> If not all dependencies can be resolved to Central itself, would that be
>> "flagged" too and also cause blocking the artifact(s) ?
>
> The validations are currently configured to disallow any release
> repository declarations in the poms. We may allow some approved
> external repos down the road if the contents are vetted and cleaned
> and unlikely to disappear. The main issues revolve around fly-by-night
> or unreliable repositories. Having these in your poms is a red herring
> and end up causing your users more harm than good.
>
>>
>> * At what stage is this policy "enforced"? I'm thinking of Apache Repository
>> when we deploy and release. Would a violation of this policy already be
>> noticed (and reported) while doing a staging release, or only at the final
>> release to Maven Central?
>
> This is enforced by the Nexus staging rules in the various forges and
> ultimately will be applied to all avenues into Central regardless of
> the source. (Rsyncs are being phased out). I have not enabled this
> rule on repository.apache.org but it is in place in most other
> locations. I wanted to be able to analyze the external repo use of
> Apache based projects and work with them before throwing down a new
> gauntlet. No panic is necessary, we'll work this out together, but
> this is a rule that is going into effect at Central and all projects,
> Apache or not will eventually have to pass the same criteria.
>
>> The latter clearly would be too little too late IMO.
>> Note: we're using Apache Repository for snapshot deployments right now, and
>> I haven't seen any "warning" about us referencing external repositories.
>
> This currently wouldn't trigger on any snapshot builds, but would
> prevent the promotion of a staged repo, exactly how you can't promote
> artifacts that aren't signed. Again, it's not enabled and I don't
> intend to enable it until I can analyze and work with projects to make
> this a non issue.
>
>>
>> * Does this new policy also affect the processing and handling of the
>> "legacy" rsync repositories at /www/people.apache.org/repo?
>
> As it will affect all sources into Central, yes this would eventually
> affect the legacy repo as well. However...
>
>> If it does, or even only partly, please let us know how and to what extend.
>> Note: we're planning a bugfix release shortly of an older version of
>> Jetspeed-2, version 2.1.4 (Apache Portals).
>> That version of Jetspeed-2 doesn't and cannot use the new Apache master pom
>> nor Apache Repository as it would require too major changes for the whole
>> project configuration itself. The current Jetspeed-2 version 2.2.0 has been
>> released through Apache Repository, and we're planning a new release 2.2.1
>> shortly too. However, for Jetspeed 2.1.4 we'll still have to use the
>> "legacy" rsync procedure.
>
> When a project is moved over to the new repo, the legacy repo is
> disabled for that project. Meaning you won't be able to deploy there
> anymore. Central can't rsync the same project from two locations and
> expect the metadata to be correct. To deploy into r.a.o, you won't
> have to use the entire new master pom, just change the url in your
> distributionManagement. Just ping me and I'll be glad to help you out
> with this.
>
>>
>> * A policy change like this will IMO affect and *restrict* any and all
>> Apache maven build based projects who want or are supposed to deploy to
>> Maven Central. *Apache* policy does not in any way restrict (maven)
>> dependencies on external repositories as long as the ASL license is honored.
>> For whatever reason, this new Maven Central policy now seems to require all
>> external dependencies be (at least also) be available from it.
>
> This affects all artifacts in Central not just Apache. We're doing it
> to improve the ecosystem, take a look at my blog referenced above and
> you'll see why this is a critical issue to be resolved.
>
>
>> What about other, generally respected and IMO also fine repositories like
>> http://download.java.net/maven/2 ?
>
> Have you actually looked at the contents there? We have and frankly
> it's a disaster. The good news is we're working to clean this up and
> get all those artifacts into Central as well.
>
> The sky isn't falling here and we aren't going to do anything to harm
> the community, this is an effort to move towards a more sustainable
> model. My answer was in response to the initial question of should
> they use external repositories and I simply wanted to point out that
> they should avoid going down a road that will have to be unwound in
> the near future.
>
> --Brian
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: Third party Maven Repository Usage
Posted by Brian Fox <br...@infinity.nu>.
>> Interesting. That's news to me... You have a pointer to more information?
>
As it turns out, almost all references to external repositories in
poms are junk or turn out to be junk after a bit of time. See here for
some examples:
http://www.sonatype.com/people/2010/03/why-external-repos-are-being-phased-out-of-central/
> * Unclear from the documentation is if this restriction on external
> repositories is limited to only the repository definitions in a pom, or if
> it is (or will be) extended to dependency resolving as well.
> If not all dependencies can be resolved to Central itself, would that be
> "flagged" too and also cause blocking the artifact(s) ?
The validations are currently configured to disallow any release
repository declarations in the poms. We may allow some approved
external repos down the road if the contents are vetted and cleaned
and unlikely to disappear. The main issues revolve around fly-by-night
or unreliable repositories. Having these in your poms is a red herring
and end up causing your users more harm than good.
>
> * At what stage is this policy "enforced"? I'm thinking of Apache Repository
> when we deploy and release. Would a violation of this policy already be
> noticed (and reported) while doing a staging release, or only at the final
> release to Maven Central?
This is enforced by the Nexus staging rules in the various forges and
ultimately will be applied to all avenues into Central regardless of
the source. (Rsyncs are being phased out). I have not enabled this
rule on repository.apache.org but it is in place in most other
locations. I wanted to be able to analyze the external repo use of
Apache based projects and work with them before throwing down a new
gauntlet. No panic is necessary, we'll work this out together, but
this is a rule that is going into effect at Central and all projects,
Apache or not will eventually have to pass the same criteria.
> The latter clearly would be too little too late IMO.
> Note: we're using Apache Repository for snapshot deployments right now, and
> I haven't seen any "warning" about us referencing external repositories.
This currently wouldn't trigger on any snapshot builds, but would
prevent the promotion of a staged repo, exactly how you can't promote
artifacts that aren't signed. Again, it's not enabled and I don't
intend to enable it until I can analyze and work with projects to make
this a non issue.
>
> * Does this new policy also affect the processing and handling of the
> "legacy" rsync repositories at /www/people.apache.org/repo?
As it will affect all sources into Central, yes this would eventually
affect the legacy repo as well. However...
> If it does, or even only partly, please let us know how and to what extend.
> Note: we're planning a bugfix release shortly of an older version of
> Jetspeed-2, version 2.1.4 (Apache Portals).
> That version of Jetspeed-2 doesn't and cannot use the new Apache master pom
> nor Apache Repository as it would require too major changes for the whole
> project configuration itself. The current Jetspeed-2 version 2.2.0 has been
> released through Apache Repository, and we're planning a new release 2.2.1
> shortly too. However, for Jetspeed 2.1.4 we'll still have to use the
> "legacy" rsync procedure.
When a project is moved over to the new repo, the legacy repo is
disabled for that project. Meaning you won't be able to deploy there
anymore. Central can't rsync the same project from two locations and
expect the metadata to be correct. To deploy into r.a.o, you won't
have to use the entire new master pom, just change the url in your
distributionManagement. Just ping me and I'll be glad to help you out
with this.
>
> * A policy change like this will IMO affect and *restrict* any and all
> Apache maven build based projects who want or are supposed to deploy to
> Maven Central. *Apache* policy does not in any way restrict (maven)
> dependencies on external repositories as long as the ASL license is honored.
> For whatever reason, this new Maven Central policy now seems to require all
> external dependencies be (at least also) be available from it.
This affects all artifacts in Central not just Apache. We're doing it
to improve the ecosystem, take a look at my blog referenced above and
you'll see why this is a critical issue to be resolved.
> What about other, generally respected and IMO also fine repositories like
> http://download.java.net/maven/2 ?
Have you actually looked at the contents there? We have and frankly
it's a disaster. The good news is we're working to clean this up and
get all those artifacts into Central as well.
The sky isn't falling here and we aren't going to do anything to harm
the community, this is an effort to move towards a more sustainable
model. My answer was in response to the initial question of should
they use external repositories and I simply wanted to point out that
they should avoid going down a road that will have to be unwound in
the near future.
--Brian
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: Third party Maven Repository Usage
Posted by Ate Douma <at...@douma.nu>.
On 03/23/2010 01:01 AM, Kevan Miller wrote:
>
> On Mar 20, 2010, at 4:45 PM, Brian Fox wrote:
>
>> At the Central repository we are restricting the inclusion of external
>> repositories because this generally creates a mess. This is being
>> enforced on new artifacts coming in so I would recommend you do not
>> add them or your artifacts themselves will end up blocked. A better
>> choice is to encourage the missing dependency projects to get their
>> stuff into Central, or find some compatible libraries that are.
>
> Interesting. That's news to me... You have a pointer to more information?
This definitely is news to me as well.
Other than the link provided by the original poster, http://maven.apache.org/guides/mini/guide-central-repository-upload.html, I can't find
much about this new policy, nor much details.
While I clearly support the goal of improving Central repository quality, this solution really caught me by (unpleasant) surprise and raises
several questions which I would like to get answered.
* Unclear from the documentation is if this restriction on external repositories is limited to only the repository definitions in a pom, or
if it is (or will be) extended to dependency resolving as well.
If not all dependencies can be resolved to Central itself, would that be "flagged" too and also cause blocking the artifact(s) ?
* At what stage is this policy "enforced"? I'm thinking of Apache Repository when we deploy and release. Would a violation of this policy
already be noticed (and reported) while doing a staging release, or only at the final release to Maven Central?
The latter clearly would be too little too late IMO.
Note: we're using Apache Repository for snapshot deployments right now, and I haven't seen any "warning" about us referencing external
repositories.
* Does this new policy also affect the processing and handling of the "legacy" rsync repositories at /www/people.apache.org/repo?
If it does, or even only partly, please let us know how and to what extend.
Note: we're planning a bugfix release shortly of an older version of Jetspeed-2, version 2.1.4 (Apache Portals).
That version of Jetspeed-2 doesn't and cannot use the new Apache master pom nor Apache Repository as it would require too major changes for
the whole project configuration itself. The current Jetspeed-2 version 2.2.0 has been released through Apache Repository, and we're planning
a new release 2.2.1 shortly too. However, for Jetspeed 2.1.4 we'll still have to use the "legacy" rsync procedure.
For both these versions we depend on a few external repositories and dependencies and having to "fix" those (shortly) really will pose a
problem for us.
* A policy change like this will IMO affect and *restrict* any and all Apache maven build based projects who want or are supposed to deploy
to Maven Central. *Apache* policy does not in any way restrict (maven) dependencies on external repositories as long as the ASL license is
honored. For whatever reason, this new Maven Central policy now seems to require all external dependencies be (at least also) be available
from it.
* Where, when, and by whom is this new policy discussed and decided upon? Or was this merely something that happened "overnight"?
As this policy affects (at least) all ASF maven based projects, its seems to me something which should not be decided within only the Maven
*project* alone.
I'm not so sure this really is desired nor if it will lead to the intended goal.
As I understand it, this policy requires *all* maven artifacts anyone (and especially ASF projects) would like to build against, to be
deployed in Maven Central. While other external repositories surely can and may exist, not so for Maven Central.
I honestly don't see how that can end up as a good solution. A single monolithic repository to "rule them all" just doesn't seem right to me.
What about other, generally respected and IMO also fine repositories like http://download.java.net/maven/2 ?
By excluding them, we're cutting off a large potential of code reuse and community benefits, or otherwise putting a large burden on those
depending on such external projects by forcing them to dual deploy to Maven Central as well.
I can imagine for some or many of those "external" repository project owners, or the users needing their artifacts, this burden is too much
to ask. Which could mean they will start ignore Maven Central all together and instead start or increase deploying (duplicating) their
artifacts elsewhere.
For Apache projects who "follow the guidelines", meaning leveraging Apache Repository and the Apache master pom configuration to streamline
and "automate" the Apache release process, going "elsewhere" is not really an option.
They will have to fix all these external (repository) dependencies, if even possible *just for the sake of Maven Central, or else indeed
"cut the cord" and go "independent"...
I really hope I'm missing the point here and none of this is going to cause much trouble after all.
Please enlighten me!
Regards,
Ate
>
> --kevan
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: Third party Maven Repository Usage
Posted by Kevan Miller <ke...@gmail.com>.
On Mar 20, 2010, at 4:45 PM, Brian Fox wrote:
> At the Central repository we are restricting the inclusion of external
> repositories because this generally creates a mess. This is being
> enforced on new artifacts coming in so I would recommend you do not
> add them or your artifacts themselves will end up blocked. A better
> choice is to encourage the missing dependency projects to get their
> stuff into Central, or find some compatible libraries that are.
Interesting. That's news to me... You have a pointer to more information?
--kevan
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Fwd: Third party Maven Repository Usage
Posted by ant elder <an...@gmail.com>.
Forwarding this as FYI. Tuscany is doing this so it sounds like we'll
need to change that soon if we're to keep putting the Tuscany
artifacts in the central Maven repository.
...ant
---------- Forwarded message ----------
From: Brian Fox <br...@infinity.nu>
Date: Sat, Mar 20, 2010 at 8:45 PM
Subject: Re: Third party Maven Repository Usage
To: general <ge...@incubator.apache.org>
At the Central repository we are restricting the inclusion of external
repositories because this generally creates a mess. This is being
enforced on new artifacts coming in so I would recommend you do not
add them or your artifacts themselves will end up blocked. A better
choice is to encourage the missing dependency projects to get their
stuff into Central, or find some compatible libraries that are.
On Mon, Mar 8, 2010 at 11:50 AM, Gurkan Erdogdu
<cg...@gmail.com> wrote:
> Hi;
>
> Is there any rule/policy for using third-party maven repositories in our
> project poms? Recently, we have required to list some repositories in
> settings.xml (for example: jboss) to our codebase built correctly. But when
> Apache-Hudson runs to built daily, it throws errors because of not finding
> required repositories. Is it OK to list those repositories in our poms?
>
> I have just found
> http://maven.apache.org/guides/mini/guide-central-repository-upload.htmldocumentation
> FAQ and common mistakes section.
>
> Thanks;
>
> --Gurkan
>
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: Third party Maven Repository Usage
Posted by Reto Bachmann-Gmuer <re...@trialox.org>.
We are having the same problem in Clerezza, one of the dependencies that
isn't in the central repo is sesame 2 [1]. I think we have the following
options:
- not include the sesame backend in the default distribution (users who need
it would have to compile it their selves of download it from elsewhere)
- repackage the whole of sesame underneath org.apache.clerezza and
distribute own artifacts (and recursively for all dependencies of sesame not
in central, tedious)
Cheers,
reto
1.
http://sourceforge.net/mailarchive/forum.php?thread_name=4B71ABD4.4080904%40apache.org&forum_name=sesame-general
On Sat, Mar 20, 2010 at 9:45 PM, Brian Fox <br...@infinity.nu> wrote:
> At the Central repository we are restricting the inclusion of external
> repositories because this generally creates a mess. This is being
> enforced on new artifacts coming in so I would recommend you do not
> add them or your artifacts themselves will end up blocked. A better
> choice is to encourage the missing dependency projects to get their
> stuff into Central, or find some compatible libraries that are.
>
> On Mon, Mar 8, 2010 at 11:50 AM, Gurkan Erdogdu
> <cg...@gmail.com> wrote:
> > Hi;
> >
> > Is there any rule/policy for using third-party maven repositories in our
> > project poms? Recently, we have required to list some repositories in
> > settings.xml (for example: jboss) to our codebase built correctly. But
> when
> > Apache-Hudson runs to built daily, it throws errors because of not
> finding
> > required repositories. Is it OK to list those repositories in our poms?
> >
> > I have just found
> >
> http://maven.apache.org/guides/mini/guide-central-repository-upload.htmldocumentation
> > FAQ and common mistakes section.
> >
> > Thanks;
> >
> > --Gurkan
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>
>
Re: Third party Maven Repository Usage
Posted by Brian Fox <br...@infinity.nu>.
At the Central repository we are restricting the inclusion of external
repositories because this generally creates a mess. This is being
enforced on new artifacts coming in so I would recommend you do not
add them or your artifacts themselves will end up blocked. A better
choice is to encourage the missing dependency projects to get their
stuff into Central, or find some compatible libraries that are.
On Mon, Mar 8, 2010 at 11:50 AM, Gurkan Erdogdu
<cg...@gmail.com> wrote:
> Hi;
>
> Is there any rule/policy for using third-party maven repositories in our
> project poms? Recently, we have required to list some repositories in
> settings.xml (for example: jboss) to our codebase built correctly. But when
> Apache-Hudson runs to built daily, it throws errors because of not finding
> required repositories. Is it OK to list those repositories in our poms?
>
> I have just found
> http://maven.apache.org/guides/mini/guide-central-repository-upload.htmldocumentation
> FAQ and common mistakes section.
>
> Thanks;
>
> --Gurkan
>
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: Third party Maven Repository Usage
Posted by Toni Menzel <to...@okidokiteam.com>.
I don't know of any apache policy regarding external repositories but i
usual don't recommend placing repositories into the POM at all.
Best practice is to use a repository manager and add the repositories there.
So, i am not sure how Apache Hudson is configured currently, but i would
suggest connecting it to a https://repository.apache.org/index.html.
(probably it already is).
Then, just make sure your desired repository is proxied there.
Toni
On Mon, Mar 8, 2010 at 4:50 PM, Gurkan Erdogdu <cg...@gmail.com>wrote:
> Hi;
>
> Is there any rule/policy for using third-party maven repositories in our
> project poms? Recently, we have required to list some repositories in
> settings.xml (for example: jboss) to our codebase built correctly. But when
> Apache-Hudson runs to built daily, it throws errors because of not finding
> required repositories. Is it OK to list those repositories in our poms?
>
> I have just found
>
> http://maven.apache.org/guides/mini/guide-central-repository-upload.htmldocumentation
> FAQ and common mistakes section.
>
> Thanks;
>
> --Gurkan
>
--
Toni Menzel
Independent Software Developer
Professional Profile: http://okidokiteam.com
toni@okidokiteam.com
http://www.ops4j.org - New Energy for OSS Communities - Open
Participation Software.