You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@maven.apache.org by Brett Porter <br...@apache.org> on 2008/03/17 00:05:43 UTC
Maven's future repository support
Forking the other thread.
Maven still needs to work properly without a repository manager (even
if it is a good practice to use one). In my opinion, that means:
- proper unique identifiers for repositories (my preference would
actually be to control this by group ID, but I see too many counter
examples in the Maven repositories to make this realistic - if anyone
has ideas on this front please say so).
- proper mirroring support (ie, specify which mirror you want to use
for central, etc so you can get a nearby one out of the box, like
CPAN, yum, etc type setups - I have some hand written notes from some
time back sitting on my desk I can kick into the wiki)
- full control over the repositories you use from the settings file.
When it comes to handling repositories in POMs - I think they should
still be in there, but only be a hint. ie, if the repo with that ID is
not configured, Maven can intelligently tell you how to configure it
if you want to, and the consequences of doing so. But I'm sure there
are plenty of other ways we could deal with this.
On top of this, explicit support for repository managers (that
supports all of them) as a way to replace one or all of your
repositories should be available and encouraged.
Are these all the use cases folks see?
- Brett
--
Brett Porter
brett@apache.org
http://blogs.exist.com/bporter/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: Maven's future repository support
Posted by Gilles Scokart <gs...@gmail.com>.
When discussing about mirroring, you could also consider
http://www.metalinker.org/
Also, the full control of the repository (or mirrors) you use is a
critical security feature. You must be sure that you can trust the
repositories you use, and the authenticity of the jar that are on this
repository.
Gilles
On 18/03/2008, Tamás Cservenák <ta...@cservenak.net> wrote:
> Hi,
>
> I always looked at repo IDs as "something local" to my machine (take
> into account Brian's note: apache.snapshot vs apache.snapshots vs
> apache-snapshot etc). How many builds exists with those variants of
> the same repo?
>
> Also, you can easily "pull in" the _same_ repo with different ID into
> the build over some dependency or different profile. What then?
>
> My thought:
>
> a) if using "entry level" / OSS maven, the repo URL will be entered by
> the user anyway.
>
> b) if using "enterprise level", where we can expect some control, the
> URL will also be added once somewhere (centrally managed settings,
> repoman or whatever), but we can expect that "enterprise level" users
> will/should use "enterprise level" tools :)
>
> Hence, the URL is the one and only thing that fulfills the HTTP
> "adressability" of repo, even if the repo is "dead" (absent, host is
> down or simply the project ceased to exist), we can keep a
> history/list/directory of know reposes and optionally offer some
> alternatives. Not saying this is a job for maven itself, mere for the
> maven community.
>
> And we can make tools easily that consumes those directories and makes the job.
>
> ~t~
>
>
> On Mon, Mar 17, 2008 at 1:33 AM, Brian E. Fox <br...@reply.infinity.nu> wrote:
> > I think there are two mutually exclusive things : 1) in an enterprise
> > with a repo man and 2) not
> >
> > So 1) If a repo manager is declared, that url is used for all lookups
> > regardless of defined repos on settings or poms. Perhaps a translation
> > to url like Nicolas' feature is used here for repo mans/proxies that
> > don't provide aggregation. This should be able to be declared in a
> > profile to enable devs to work in an enterprise/with repo man and
> > without easily (currently it's a pain to switch back and forth)
> >
> > 2) This is where proxies/mirrors/repo definitions come into play. Same
> > as above, all should be in profiles. I think that mirroring by Id isn't
> > always the greatest, but I also see no harm in continuing to support it
> > because it can be useful in some instances. The adhoc nature of
> > declaring repos is annoying to be sure (I'm sure everyone has seen
> > apache.snapshot and apache.snapshots) but I don't currently have any
> > ideas how this can be handled better.
> >
> > An additional mirrorOfUrl could be added to the settings so you could
> > use mirrors by id or by url as you see fit.
> >
> > It would be nice to provide some automatic geo lookup but I'm not sure
> > how that would happen. It seems like this data needs to be stored in the
> > repo itself and then cached in the local repo. Otherwise someone would
> > have to provide a redirection service which isn't feasible for all
> > repos.
> >
> >
> >
> >
> > -----Original Message-----
> > From: Brett Porter [mailto:brett@apache.org]
> > Sent: Sunday, March 16, 2008 7:06 PM
> > To: Maven Developers List
> > Subject: Maven's future repository support
> >
> > Forking the other thread.
> >
> > Maven still needs to work properly without a repository manager (even
> > if it is a good practice to use one). In my opinion, that means:
> > - proper unique identifiers for repositories (my preference would
> > actually be to control this by group ID, but I see too many counter
> > examples in the Maven repositories to make this realistic - if anyone
> > has ideas on this front please say so).
> > - proper mirroring support (ie, specify which mirror you want to use
> > for central, etc so you can get a nearby one out of the box, like
> > CPAN, yum, etc type setups - I have some hand written notes from some
> > time back sitting on my desk I can kick into the wiki)
> > - full control over the repositories you use from the settings file.
> >
> > When it comes to handling repositories in POMs - I think they should
> > still be in there, but only be a hint. ie, if the repo with that ID is
> > not configured, Maven can intelligently tell you how to configure it
> > if you want to, and the consequences of doing so. But I'm sure there
> > are plenty of other ways we could deal with this.
> >
> > On top of this, explicit support for repository managers (that
> > supports all of them) as a way to replace one or all of your
> > repositories should be available and encouraged.
> >
> > Are these all the use cases folks see?
> >
> > - Brett
> >
> > --
> > Brett Porter
> > brett@apache.org
> > http://blogs.exist.com/bporter/
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > For additional commands, e-mail: dev-help@maven.apache.org
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > For additional commands, e-mail: dev-help@maven.apache.org
> >
> >
>
>
>
>
> --
> Thanks,
>
> ~t~
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
--
Gilles Scokart
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: Maven's future repository support
Posted by Tamás Cservenák <ta...@cservenak.net>.
Hi,
I always looked at repo IDs as "something local" to my machine (take
into account Brian's note: apache.snapshot vs apache.snapshots vs
apache-snapshot etc). How many builds exists with those variants of
the same repo?
Also, you can easily "pull in" the _same_ repo with different ID into
the build over some dependency or different profile. What then?
My thought:
a) if using "entry level" / OSS maven, the repo URL will be entered by
the user anyway.
b) if using "enterprise level", where we can expect some control, the
URL will also be added once somewhere (centrally managed settings,
repoman or whatever), but we can expect that "enterprise level" users
will/should use "enterprise level" tools :)
Hence, the URL is the one and only thing that fulfills the HTTP
"adressability" of repo, even if the repo is "dead" (absent, host is
down or simply the project ceased to exist), we can keep a
history/list/directory of know reposes and optionally offer some
alternatives. Not saying this is a job for maven itself, mere for the
maven community.
And we can make tools easily that consumes those directories and makes the job.
~t~
On Mon, Mar 17, 2008 at 1:33 AM, Brian E. Fox <br...@reply.infinity.nu> wrote:
> I think there are two mutually exclusive things : 1) in an enterprise
> with a repo man and 2) not
>
> So 1) If a repo manager is declared, that url is used for all lookups
> regardless of defined repos on settings or poms. Perhaps a translation
> to url like Nicolas' feature is used here for repo mans/proxies that
> don't provide aggregation. This should be able to be declared in a
> profile to enable devs to work in an enterprise/with repo man and
> without easily (currently it's a pain to switch back and forth)
>
> 2) This is where proxies/mirrors/repo definitions come into play. Same
> as above, all should be in profiles. I think that mirroring by Id isn't
> always the greatest, but I also see no harm in continuing to support it
> because it can be useful in some instances. The adhoc nature of
> declaring repos is annoying to be sure (I'm sure everyone has seen
> apache.snapshot and apache.snapshots) but I don't currently have any
> ideas how this can be handled better.
>
> An additional mirrorOfUrl could be added to the settings so you could
> use mirrors by id or by url as you see fit.
>
> It would be nice to provide some automatic geo lookup but I'm not sure
> how that would happen. It seems like this data needs to be stored in the
> repo itself and then cached in the local repo. Otherwise someone would
> have to provide a redirection service which isn't feasible for all
> repos.
>
>
>
>
> -----Original Message-----
> From: Brett Porter [mailto:brett@apache.org]
> Sent: Sunday, March 16, 2008 7:06 PM
> To: Maven Developers List
> Subject: Maven's future repository support
>
> Forking the other thread.
>
> Maven still needs to work properly without a repository manager (even
> if it is a good practice to use one). In my opinion, that means:
> - proper unique identifiers for repositories (my preference would
> actually be to control this by group ID, but I see too many counter
> examples in the Maven repositories to make this realistic - if anyone
> has ideas on this front please say so).
> - proper mirroring support (ie, specify which mirror you want to use
> for central, etc so you can get a nearby one out of the box, like
> CPAN, yum, etc type setups - I have some hand written notes from some
> time back sitting on my desk I can kick into the wiki)
> - full control over the repositories you use from the settings file.
>
> When it comes to handling repositories in POMs - I think they should
> still be in there, but only be a hint. ie, if the repo with that ID is
> not configured, Maven can intelligently tell you how to configure it
> if you want to, and the consequences of doing so. But I'm sure there
> are plenty of other ways we could deal with this.
>
> On top of this, explicit support for repository managers (that
> supports all of them) as a way to replace one or all of your
> repositories should be available and encouraged.
>
> Are these all the use cases folks see?
>
> - Brett
>
> --
> Brett Porter
> brett@apache.org
> http://blogs.exist.com/bporter/
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
--
Thanks,
~t~
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
RE: Maven's future repository support
Posted by "Brian E. Fox" <br...@reply.infinity.nu>.
I think there are two mutually exclusive things : 1) in an enterprise
with a repo man and 2) not
So 1) If a repo manager is declared, that url is used for all lookups
regardless of defined repos on settings or poms. Perhaps a translation
to url like Nicolas' feature is used here for repo mans/proxies that
don't provide aggregation. This should be able to be declared in a
profile to enable devs to work in an enterprise/with repo man and
without easily (currently it's a pain to switch back and forth)
2) This is where proxies/mirrors/repo definitions come into play. Same
as above, all should be in profiles. I think that mirroring by Id isn't
always the greatest, but I also see no harm in continuing to support it
because it can be useful in some instances. The adhoc nature of
declaring repos is annoying to be sure (I'm sure everyone has seen
apache.snapshot and apache.snapshots) but I don't currently have any
ideas how this can be handled better.
An additional mirrorOfUrl could be added to the settings so you could
use mirrors by id or by url as you see fit.
It would be nice to provide some automatic geo lookup but I'm not sure
how that would happen. It seems like this data needs to be stored in the
repo itself and then cached in the local repo. Otherwise someone would
have to provide a redirection service which isn't feasible for all
repos.
-----Original Message-----
From: Brett Porter [mailto:brett@apache.org]
Sent: Sunday, March 16, 2008 7:06 PM
To: Maven Developers List
Subject: Maven's future repository support
Forking the other thread.
Maven still needs to work properly without a repository manager (even
if it is a good practice to use one). In my opinion, that means:
- proper unique identifiers for repositories (my preference would
actually be to control this by group ID, but I see too many counter
examples in the Maven repositories to make this realistic - if anyone
has ideas on this front please say so).
- proper mirroring support (ie, specify which mirror you want to use
for central, etc so you can get a nearby one out of the box, like
CPAN, yum, etc type setups - I have some hand written notes from some
time back sitting on my desk I can kick into the wiki)
- full control over the repositories you use from the settings file.
When it comes to handling repositories in POMs - I think they should
still be in there, but only be a hint. ie, if the repo with that ID is
not configured, Maven can intelligently tell you how to configure it
if you want to, and the consequences of doing so. But I'm sure there
are plenty of other ways we could deal with this.
On top of this, explicit support for repository managers (that
supports all of them) as a way to replace one or all of your
repositories should be available and encouraged.
Are these all the use cases folks see?
- Brett
--
Brett Porter
brett@apache.org
http://blogs.exist.com/bporter/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: Maven's future repository support
Posted by Nigel Magnay <ni...@gmail.com>.
That sounds good to me, and I see where the identifiers have come in;
I think you could do without them though, and just use the repository
URI as the identifier :-
For 'external' mirrors, accessed directly and not through a repo manager:
Say you have repo1.maven.org/maven2, and global mirrors of
eu.repo1.maven.org/maven2 and us.repo1.maven.org/maven2. I think the
<repository> doesn't need an id. There are a few options to get it to
use a more localised mirror
- Geo aware DNS (I think things like wikipedia are doing this). A bit hard core.
- Your settings.xml file, containing a list of mirrors/redirect
information based on the URI. But this settings.xml could be
configured from choices either shipped with maven after installation,
or (better), there might be a repo-config-SNAPSHOT.xml[1] in
repo1.maven.org/maven2/repo1/maven/org/maven2/ that contains this
information. Something interactive gives you the choice of
manipulating your settings from the known, valid options. But a
default, non-interactive build would still know where to go.
For 'internal' mirrors, accessed through a repo manager:
- Your install wants http://repo1.maven.org/maven2
- Your settings, as you're inside the corp firewall, say 'here is the
proxy server'
- The repo manager uses the appropriate strategy (including the above)
to fetch or deny from repo1 or the appropriate mirror.
[1] Or whatever. What would be really nice is that as well as keeping
metadata about artifacts in the repository, there was a way of
accessing (optionally available) metadata about the repository itself
in some downloadable artifact. That way an entire 'uptodate' check for
repo1 might be simply checking the ETag on 1 file. A black/whitelist
of available artifacts would help preventing asking for SNAPSHOT
artifacts from umpteen repos before hitting the right one; if
search/index information were standardised, that could be transferred
as well...
On Sun, Mar 16, 2008 at 11:05 PM, Brett Porter <br...@apache.org> wrote:
> Forking the other thread.
>
> Maven still needs to work properly without a repository manager (even
> if it is a good practice to use one). In my opinion, that means:
> - proper unique identifiers for repositories (my preference would
> actually be to control this by group ID, but I see too many counter
> examples in the Maven repositories to make this realistic - if anyone
> has ideas on this front please say so).
> - proper mirroring support (ie, specify which mirror you want to use
> for central, etc so you can get a nearby one out of the box, like
> CPAN, yum, etc type setups - I have some hand written notes from some
> time back sitting on my desk I can kick into the wiki)
> - full control over the repositories you use from the settings file.
>
> When it comes to handling repositories in POMs - I think they should
> still be in there, but only be a hint. ie, if the repo with that ID is
> not configured, Maven can intelligently tell you how to configure it
> if you want to, and the consequences of doing so. But I'm sure there
> are plenty of other ways we could deal with this.
>
> On top of this, explicit support for repository managers (that
> supports all of them) as a way to replace one or all of your
> repositories should be available and encouraged.
>
> Are these all the use cases folks see?
>
> - Brett
>
> --
> Brett Porter
> brett@apache.org
> http://blogs.exist.com/bporter/
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org