You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@maven.apache.org by Mark Derricutt <ma...@talios.com> on 2014/06/26 14:34:15 UTC
POM 5.0 and Maven.next idea - re: 's
In last weeks dev hangout I raised the idea of removing <repository>
elements due to some issues with them regarding mirrors etc which was
somewhat negatively received, however I've been thinking about this a
bit and came up with an interesting idea earlier in the night whilst at
a gig.
One of the problems I'm often seeing is that:
1) a project uploads their artefact to a repository ( mostly maven
central )
2) 90% of their dependencies are available from the repository in
question
3) 1 critical dependency is not - which ultimately means you can't
build......if you have a mirror setup
(usually because you've not noticed a <repository> declaration which you
need to configure in your nexus/arifactory/archiva etc. )
The idea I had is three fold:
1) Fallback on original repository when a mirror doesn't resolve
When maven is checking for a repository for an artefact, and using a
mirror - if that artefact can't be found, maven should retry using the
original repository directly with builds warnings.
2) Deploy transitive runtime dependencies along with your release
We currently have the <distributionManagement> section of a pom
declaring the deploy repository. If we added a new `deploy-dependencies`
goal to some plugin and updated maven-release-plugin with a this in its
default deployment goals, I think some magical things could happen:
- Find the _runtime_ dependency tree of your project
- Check which artefacts don't exist on the deployment repo
- Deploy those artefacts out to the repository - essentially an
implicit mirroring your dependencies.
At the same time, modify the POM reader to add the repository defined in
`<distributionManagement>` as a `<repository>` for discovering
dependencies.
Any thoughts on this? Since it's now 12:32am, _hopefully_ I'll be awake
in time to join the discussion tomorrow to raise this idea.
Cheers,
Mark
Re: POM 5.0 and Maven.next idea - re: 's
Posted by Mark Derricutt <ma...@talios.com>.
Hi Jörg,
On 27 Jun 2014, at 2:11, Jörg Schaible wrote:
>> When maven is checking for a repository for an artefact, and using a
>> mirror - if that artefact can't be found, maven should retry using
>> the
>> original repository directly with builds warnings.
>
> Very bad idea. Especially if the original repo is not/no longer
> reachable
> and you have to wait for a timeout. And you can no longer control with
> an
> Archive Manager what actually is used from where.
True - as mentioned in one of my other replies, this was the part of the
idea I wasn't 100% sold on myself, and the reason you state here is also
the #1 reason I loathe <repository> being mentioned inside a pom.xml as
well.
>> essentially an
>> implicit mirroring your dependencies.
> And suddenly we have all kind of stuff in public repos that are not
> allowed
> to be distributed.
True - a further thought on this was that a) `optional` or `provided`
dependencies shouldn't be published, but further - we _could_ add an
`exclusions` list inside `dependencyManagement` to exclude those that
shouldn't be published.
Part of my thinking here is that if you're artefact depends on things
that can't be published, why are you pushing that artefact to a public
repository in the first place?
> IMHO, the best option *is* actually to remove the <repository>
> elements.
Agreed - but for other reasons.
Re: POM 5.0 and Maven.next idea - re: 's
Posted by Christian Schulte <cs...@schulte.it>.
Am 06/26/14 17:18, schrieb Stephen Connolly:
> And the staleness is another killer on the: "oh why don't we use DNS
> records to resolve the repository" because what if foobar.org do not renew
> their domain, now all the org.foobar artifacts will just drop off the net
> and we have no means to inject a repo for them again...
Central authority fallback zone. So if 'foobar.org' isn't renewed so
that DNS becomes unavailable, one could move that groupId to a fallback
zone like: 'groupid.central.apache.org' or something like this. Some
mechanism like this would already be required for group ids like
'commons-io' or such which cannot be mapped to a domain directly. So
resolve repositories for 'groupId' by DNS at 'groupId', not finding any
records there, search the fallback zone 'groupId.central.apache.org',
not finding anything there, fallback to legacy POM repositories
(optionally). Still nothing found, fail with artifact not found. If a
'<repository>' with an URL like 'repository.foobar.org' is used, that
repository would become stale as well already. You would have control
about that via such a repository authority zone.
> I think dropping <repositories> and using social pressure to "get thee to
> central" is probably the best worst long term solution
Nothing can stop people from grabbing the Maven source distribution and
patching it to theire needs and nothing can stop people from grabbing
the sources of some artifact from 'central' and redeploy it to some
other repository.
I am heading after a mechanism to make the artifact providers the
autoritative entity controlling the repositories to resolve _theire_
artifacts from. If someone needs to e.g. deploy a custom 'commons-io',
they are forced to change the groupId to something they are
authoritative for since Maven will never fetch that 'commons-io' from a
non-authoritative repository. Just put everything into 'central' isn't
practical, I think. That's like deploying all websites of the world to a
central server. You type 'maven.apache.org' into the browser and you are
connected to the corresponding web server. You type
'commons-io:commons-io:2.4' into the POM, and you get connected to
somewhere non-authoritative and fetch some non-authoritative content, if
not careful and the providers of 'commons-io' cannot do anything about that.
Regards,
--
Christian
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: POM 5.0 and Maven.next idea - re: 's
Posted by Mark Derricutt <ma...@talios.com>.
On 27 Jun 2014, at 3:18, Stephen Connolly wrote:
> I think dropping <repositories> and using social pressure to "get thee to
> central" is probably the best worst long term solution
Best worst?
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: POM 5.0 and Maven.next idea - re: 's
Posted by Stephen Connolly <st...@gmail.com>.
And the staleness is another killer on the: "oh why don't we use DNS
records to resolve the repository" because what if foobar.org do not renew
their domain, now all the org.foobar artifacts will just drop off the net
and we have no means to inject a repo for them again...
I think dropping <repositories> and using social pressure to "get thee to
central" is probably the best worst long term solution
On 26 June 2014 15:18, Paul Benedict <pb...@apache.org> wrote:
> I agree with Jorg. Furthermore, I have found repositories addresses change
> overtime. I *don't* want this information in the POM either because it
> becomes stale. It's not really build information, per se, as it is Maven
> connection information. I have to fallback to modifying my settings.xml
> when I encounter one of these situations -- which is "smell" enough of this
> very issue.
>
>
>
> Cheers,
> Paul
>
>
> On Thu, Jun 26, 2014 at 9:11 AM, Jörg Schaible <
> joerg.schaible@swisspost.com
> > wrote:
>
> > Hi Mark,
> >
> > Mark Derricutt wrote:
> >
> > > In last weeks dev hangout I raised the idea of removing <repository>
> > > elements due to some issues with them regarding mirrors etc which was
> > > somewhat negatively received, however I've been thinking about this a
> > > bit and came up with an interesting idea earlier in the night whilst at
> > > a gig.
> > >
> > > One of the problems I'm often seeing is that:
> > >
> > > 1) a project uploads their artefact to a repository ( mostly maven
> > > central )
> > > 2) 90% of their dependencies are available from the repository in
> > > question
> > > 3) 1 critical dependency is not - which ultimately means you can't
> > > build......if you have a mirror setup
> > >
> > > (usually because you've not noticed a <repository> declaration which
> you
> > > need to configure in your nexus/arifactory/archiva etc. )
> > >
> > > The idea I had is three fold:
> > >
> > > 1) Fallback on original repository when a mirror doesn't resolve
> > >
> > > When maven is checking for a repository for an artefact, and using a
> > > mirror - if that artefact can't be found, maven should retry using the
> > > original repository directly with builds warnings.
> >
> >
> > Very bad idea. Especially if the original repo is not/no longer reachable
> > and you have to wait for a timeout. And you can no longer control with an
> > Archive Manager what actually is used from where.
> >
> >
> > > 2) Deploy transitive runtime dependencies along with your release
> > >
> > > We currently have the <distributionManagement> section of a pom
> > > declaring the deploy repository. If we added a new
> `deploy-dependencies`
> > > goal to some plugin and updated maven-release-plugin with a this in its
> > > default deployment goals, I think some magical things could happen:
> > >
> > > - Find the _runtime_ dependency tree of your project
> > > - Check which artefacts don't exist on the deployment repo
> > > - Deploy those artefacts out to the repository - essentially an
> > > implicit mirroring your dependencies.
> > >
> > > At the same time, modify the POM reader to add the repository defined
> in
> > > `<distributionManagement>` as a `<repository>` for discovering
> > > dependencies.
> > >
> > > Any thoughts on this? Since it's now 12:32am, _hopefully_ I'll be awake
> > > in time to join the discussion tomorrow to raise this idea.
> >
> >
> > And suddenly we have all kind of stuff in public repos that are not
> allowed
> > to be distributed.
> >
> > IMHO, the best option *is* actually to remove the <repository> elements.
> >
> > - Jörg
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > For additional commands, e-mail: dev-help@maven.apache.org
> >
> >
>
Re: POM 5.0 and Maven.next idea - re: 's
Posted by Mark Derricutt <ma...@talios.com>.
On 27 Jun 2014, at 2:18, Paul Benedict wrote:
> I agree with Jorg. Furthermore, I have found repositories addresses
> change
> overtime. I *don't* want this information in the POM either because it
> becomes stale. It's not really build information, per se, as it is
> Maven
> connection information. I have to fallback to modifying my
> settings.xml
> when I encounter one of these situations -- which is "smell" enough of
> this
> very issue.
+1 - it's this reason I and others recommend all devs run a local
archiva/nexus as a mirror, even if they have a company mirror - then you
can add such repositories either company wide, or local if you need
them.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: POM 5.0 and Maven.next idea - re: 's
Posted by Paul Benedict <pb...@apache.org>.
I agree with Jorg. Furthermore, I have found repositories addresses change
overtime. I *don't* want this information in the POM either because it
becomes stale. It's not really build information, per se, as it is Maven
connection information. I have to fallback to modifying my settings.xml
when I encounter one of these situations -- which is "smell" enough of this
very issue.
Cheers,
Paul
On Thu, Jun 26, 2014 at 9:11 AM, Jörg Schaible <joerg.schaible@swisspost.com
> wrote:
> Hi Mark,
>
> Mark Derricutt wrote:
>
> > In last weeks dev hangout I raised the idea of removing <repository>
> > elements due to some issues with them regarding mirrors etc which was
> > somewhat negatively received, however I've been thinking about this a
> > bit and came up with an interesting idea earlier in the night whilst at
> > a gig.
> >
> > One of the problems I'm often seeing is that:
> >
> > 1) a project uploads their artefact to a repository ( mostly maven
> > central )
> > 2) 90% of their dependencies are available from the repository in
> > question
> > 3) 1 critical dependency is not - which ultimately means you can't
> > build......if you have a mirror setup
> >
> > (usually because you've not noticed a <repository> declaration which you
> > need to configure in your nexus/arifactory/archiva etc. )
> >
> > The idea I had is three fold:
> >
> > 1) Fallback on original repository when a mirror doesn't resolve
> >
> > When maven is checking for a repository for an artefact, and using a
> > mirror - if that artefact can't be found, maven should retry using the
> > original repository directly with builds warnings.
>
>
> Very bad idea. Especially if the original repo is not/no longer reachable
> and you have to wait for a timeout. And you can no longer control with an
> Archive Manager what actually is used from where.
>
>
> > 2) Deploy transitive runtime dependencies along with your release
> >
> > We currently have the <distributionManagement> section of a pom
> > declaring the deploy repository. If we added a new `deploy-dependencies`
> > goal to some plugin and updated maven-release-plugin with a this in its
> > default deployment goals, I think some magical things could happen:
> >
> > - Find the _runtime_ dependency tree of your project
> > - Check which artefacts don't exist on the deployment repo
> > - Deploy those artefacts out to the repository - essentially an
> > implicit mirroring your dependencies.
> >
> > At the same time, modify the POM reader to add the repository defined in
> > `<distributionManagement>` as a `<repository>` for discovering
> > dependencies.
> >
> > Any thoughts on this? Since it's now 12:32am, _hopefully_ I'll be awake
> > in time to join the discussion tomorrow to raise this idea.
>
>
> And suddenly we have all kind of stuff in public repos that are not allowed
> to be distributed.
>
> IMHO, the best option *is* actually to remove the <repository> elements.
>
> - Jörg
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
Re: POM 5.0 and Maven.next idea - re: 's
Posted by Jörg Schaible <jo...@swisspost.com>.
Hi Mark,
Mark Derricutt wrote:
> In last weeks dev hangout I raised the idea of removing <repository>
> elements due to some issues with them regarding mirrors etc which was
> somewhat negatively received, however I've been thinking about this a
> bit and came up with an interesting idea earlier in the night whilst at
> a gig.
>
> One of the problems I'm often seeing is that:
>
> 1) a project uploads their artefact to a repository ( mostly maven
> central )
> 2) 90% of their dependencies are available from the repository in
> question
> 3) 1 critical dependency is not - which ultimately means you can't
> build......if you have a mirror setup
>
> (usually because you've not noticed a <repository> declaration which you
> need to configure in your nexus/arifactory/archiva etc. )
>
> The idea I had is three fold:
>
> 1) Fallback on original repository when a mirror doesn't resolve
>
> When maven is checking for a repository for an artefact, and using a
> mirror - if that artefact can't be found, maven should retry using the
> original repository directly with builds warnings.
Very bad idea. Especially if the original repo is not/no longer reachable
and you have to wait for a timeout. And you can no longer control with an
Archive Manager what actually is used from where.
> 2) Deploy transitive runtime dependencies along with your release
>
> We currently have the <distributionManagement> section of a pom
> declaring the deploy repository. If we added a new `deploy-dependencies`
> goal to some plugin and updated maven-release-plugin with a this in its
> default deployment goals, I think some magical things could happen:
>
> - Find the _runtime_ dependency tree of your project
> - Check which artefacts don't exist on the deployment repo
> - Deploy those artefacts out to the repository - essentially an
> implicit mirroring your dependencies.
>
> At the same time, modify the POM reader to add the repository defined in
> `<distributionManagement>` as a `<repository>` for discovering
> dependencies.
>
> Any thoughts on this? Since it's now 12:32am, _hopefully_ I'll be awake
> in time to join the discussion tomorrow to raise this idea.
And suddenly we have all kind of stuff in public repos that are not allowed
to be distributed.
IMHO, the best option *is* actually to remove the <repository> elements.
- Jörg
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: POM 5.0 and Maven.next idea - re: 's
Posted by Mark Derricutt <ma...@talios.com>.
On 27 Jun 2014, at 7:27, Michael Osipov wrote:
> Not going to work if you are behing a MRM instance and proxied in a
> company like me.
Given that <mirrors> is a settings.xml thing, thats purely your local
maven installation - if we're talking about changing maven, that
potentially could be up for changes.
That being said - this was the only part of my idea I thought was
horrible and not overly worth pursuing.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: POM 5.0 and Maven.next idea - re: 's
Posted by Mark Derricutt <ma...@talios.com>.
On 27 Jun 2014, at 7:47, Bernd Eckenfels wrote:
>>>> 2) Deploy transitive runtime dependencies along with your release
>
> ... or make sure they are centrally available. I think this is an
> (important) best practice since years:
+1 - that might even be a good first step - and could even just be a
plugin in its own right - maven-central-plugin:check-dependencies :)
>
> http://blog.sonatype.com/2010/03/why-external-repos-are-being-phased-out-of-central/
>
> And it is really good if it is enforced on MC.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: POM 5.0 and Maven.next idea - re: 's
Posted by Bernd Eckenfels <ec...@zusammenkunft.net>.
Am Thu, 26 Jun 2014 21:44:55 +0200
schrieb Michael Osipov <mi...@apache.org>:
> Am 2014-06-26 21:41, schrieb Mark Derricutt:
> > On 27 Jun 2014, at 7:27, Michael Osipov wrote:
> >
> >> 2) Deploy transitive runtime dependencies along with your release
... or make sure they are centrally available. I think this is an
(important) best practice since years:
http://blog.sonatype.com/2010/03/why-external-repos-are-being-phased-out-of-central/
And it is really good if it is enforced on MC.
Gruss
Bernd
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: POM 5.0 and Maven.next idea - re: 's
Posted by Mark Derricutt <ma...@talios.com>.
On 27 Jun 2014, at 7:44, Michael Osipov wrote:
> let me rephrase your intention: you want to re-upload dependencies --
> even if they are already in a repo?!
> Am I wrong?
Essentially yes - as a means of enforcing that "everything required to
actually compile or use my artefact is in a repository", that way
limiting the need for consumers having to scour and configure N number
of mirrored repositories in a nexus or something.
Mostly as some form of "guarantee of accessibility", and in the name of
reproducible builds.
amrk
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: POM 5.0 and Maven.next idea - re: 's
Posted by Michael Osipov <mi...@apache.org>.
Am 2014-06-26 21:41, schrieb Mark Derricutt:
> On 27 Jun 2014, at 7:27, Michael Osipov wrote:
>
>> 2) Deploy transitive runtime dependencies along with your release
>>
>>
>> This beats DRY and reinvents the wheel.
>>
>> I would obstain doing either one.
>
> I don't see this as repeating oneself, just about populating a
> repository with required dependencies - not bundling dependencies inside
> your artefact or anything.
>
> Still - it's only an idea for discussion around a common problem.
Mark,
let me rephrase your intention: you want to re-upload dependencies --
even if they are already in a repo?!
Am I wrong?
Michael
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: POM 5.0 and Maven.next idea - re: 's
Posted by Mark Derricutt <ma...@talios.com>.
On 27 Jun 2014, at 7:27, Michael Osipov wrote:
> 2) Deploy transitive runtime dependencies along with your release
>
>
> This beats DRY and reinvents the wheel.
>
> I would obstain doing either one.
I don't see this as repeating oneself, just about populating a
repository with required dependencies - not bundling dependencies inside
your artefact or anything.
Still - it's only an idea for discussion around a common problem.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: POM 5.0 and Maven.next idea - re: 's
Posted by Michael Osipov <mi...@apache.org>.
Am 2014-06-26 14:34, schrieb Mark Derricutt:
> In last weeks dev hangout I raised the idea of removing <repository>
> elements due to some issues with them regarding mirrors etc which was
> somewhat negatively received, however I've been thinking about this a
> bit and came up with an interesting idea earlier in the night whilst at
> a gig.
>
> One of the problems I'm often seeing is that:
>
> 1) a project uploads their artefact to a repository ( mostly maven
> central )
> 2) 90% of their dependencies are available from the repository in question
> 3) 1 critical dependency is not - which ultimately means you can't
> build......if you have a mirror setup
>
> (usually because you've not noticed a <repository> declaration which you
> need to configure in your nexus/arifactory/archiva etc. )
>
> The idea I had is three fold:
>
> 1) Fallback on original repository when a mirror doesn't resolve
>
> When maven is checking for a repository for an artefact, and using a
> mirror - if that artefact can't be found, maven should retry using the
> original repository directly with builds warnings.
Not going to work if you are behing a MRM instance and proxied in a
company like me.
> 2) Deploy transitive runtime dependencies along with your release
This beats DRY and reinvents the wheel.
I would obstain doing either one.
Michael
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: POM 5.0 and Maven.next idea - re: 's
Posted by Mark Derricutt <ma...@talios.com>.
On 27 Jun 2014, at 0:55, Arnaud Héritier wrote:
> My main concern with current POM entries about repositories is that we are
> hardcoding some information that may change in the future ( the repo moved,
> sources moved ...). Thus depending for what you are looking at an old POM
> it may be a problem or not.
Even including those mentioned in distributionManagement?
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: POM 5.0 and Maven.next idea - re: 's
Posted by Arnaud Héritier <ah...@gmail.com>.
Hi,
On Thu, Jun 26, 2014 at 2:34 PM, Mark Derricutt <ma...@talios.com> wrote:
> In last weeks dev hangout I raised the idea of removing <repository>
> elements due to some issues with them regarding mirrors etc which was
> somewhat negatively received, however I've been thinking about this a bit
> and came up with an interesting idea earlier in the night whilst at a gig.
>
> One of the problems I'm often seeing is that:
>
> 1) a project uploads their artefact to a repository ( mostly maven central
> )
> 2) 90% of their dependencies are available from the repository in question
> 3) 1 critical dependency is not - which ultimately means you can't
> build......if you have a mirror setup
>
> (usually because you've not noticed a <repository> declaration which you
> need to configure in your nexus/arifactory/archiva etc. )
>
> The idea I had is three fold:
>
> 1) Fallback on original repository when a mirror doesn't resolve
>
> When maven is checking for a repository for an artefact, and using a
> mirror - if that artefact can't be found, maven should retry using the
> original repository directly with builds warnings.
>
Don't forget here that some companies will want to enforce (for various
reasons) to use specific repository and only it.
>
> 2) Deploy transitive runtime dependencies along with your release
>
> We currently have the <distributionManagement> section of a pom declaring
> the deploy repository. If we added a new `deploy-dependencies` goal to some
> plugin and updated maven-release-plugin with a this in its default
> deployment goals, I think some magical things could happen:
>
> - Find the _runtime_ dependency tree of your project
> - Check which artefacts don't exist on the deployment repo
> - Deploy those artefacts out to the repository - essentially an implicit
> mirroring your dependencies.
>
> At the same time, modify the POM reader to add the repository defined in
> `<distributionManagement>` as a `<repository>` for discovering dependencies.
>
My main concern with current POM entries about repositories is that we are
hardcoding some information that may change in the future ( the repo moved,
sources moved ...). Thus depending for what you are looking at an old POM
it may be a problem or not.
>
> Any thoughts on this? Since it's now 12:32am, _hopefully_ I'll be awake in
> time to join the discussion tomorrow to raise this idea.
>
> Cheers,
> Mark
>
cheers
--
-----
Arnaud Héritier
http://aheritier.net
Mail/GTalk: aheritier AT gmail DOT com
Twitter/Skype : aheritier