You are viewing a plain text version of this content. The canonical link for it is here.
Posted to repository@apache.org by Patrick Chanezon <pa...@chanezon.com> on 2004/02/03 04:11:33 UTC
cool uris don't change, don't they ?
Hi all,
I've been lurking in that list (repository) for a while, because the
topic of repositories is of interest to me.
I think that maven will have a big effect on java development and I
understand that the apache repository will be where maven stores its
jars (not sure about that but that would make sense, cross posting to
maven-dev).
I just wanted to check if you quantified or considered the requirement
about how long you would keep artifacts at their url.
Today I experienced some broken build with an unofficial makefile that
helps build a system called Syncato (details at
http://www.chanezon.com/pat/weblog/archives/000131.html).
Rick Bradley's syncatomatic is a hack, and it broke because it made
assumptions about xerces 2.3 source distribution to be hosted on an
apache mirror.
I thought: no big deal for a quick hack like this, designed to help
people out temporarily.
But when most java software gets built using maven, I guess you don't
want that kind of things to happen. And it could happen all over the
place, in a few years, if old version of artifacts get decommissionned
wihtout warning.
Did you specify a lifecycle for artifacts, with some durations, and a
process to decommision them ?
Also maybe when decommisioning time approaches, the server could notify
maven, and maven could issue some warnings, pretty much like deprecation
messages in java compilation, so that developers get warned that they
should upgrade to a newer library.
P@
Re: cool uris don't change, don't they ?
Posted by Patrick Chanezon <pa...@chanezon.com>.
Thanks, I think this covers it: latest version of each point release on
the mirrors, and an automatic decommissionning based on usage for older
ones, with redirection to archives.
P@
Nick Chalko wrote:
> Good point.
> I think in general mirrors should be able to choose whatever subset
> they want.
> But apache should maintain at lest "the latest version of each point
> release"
>
> Mark R. Diggory wrote:
>
>> With the amount of versioning going on, eventually a release falls
>> into a state of non-usage, I suspect there should be room for such a
>> mechanism, otherwise mirrors will become bloated with unused,
>> outdated, antiquated and obsolete content.
>>
>> I suspect some sort of "redirect" mechanism would be sufficient in
>> cases where an unmirrored archive is used. Something that most web
>> servers support (for instance Apache Httpd and .htaccess files)
>>
>> Existing Example:
>>
>> archives.apache.org represents content from www.apache.org/dist that
>> has been "decommissioned" from the mirroring process, of course
>> mirrors may maintain copies of these files by not deleting contents.
>>
>> Ideally, such a mechanism could even be automated based on historical
>> download information on the resource. I.E. if the resource hasn't
>> been downloaded in 5 years, move it into an archive and provide a
>> redirect or notice.
>>
>> -Mark
>>
>> Nick Chalko wrote:
>>
>>> Patrick Chanezon wrote:
>>>
>>>> Did you specify a lifecycle for artifacts, with some durations, and
>>>> a process to decommision them ?
>>>>
>>>
>>> Good question. This may be something to put to the board. My
>>> general thought are. "Released" version should live forever, unless
>>> a security or other fatal flaw is found in a release.
>>>
>>> As a minimum I think the latest version of each point release should
>>> be kept ie 1.2.x.
>>> R,
>>> Nick
>>>
>>
>
>
>
>
Re: cool uris don't change, don't they ?
Posted by Nick Chalko <ni...@chalko.com>.
Good point.
I think in general mirrors should be able to choose whatever subset they
want.
But apache should maintain at lest "the latest version of each point
release"
Mark R. Diggory wrote:
> With the amount of versioning going on, eventually a release falls
> into a state of non-usage, I suspect there should be room for such a
> mechanism, otherwise mirrors will become bloated with unused,
> outdated, antiquated and obsolete content.
>
> I suspect some sort of "redirect" mechanism would be sufficient in
> cases where an unmirrored archive is used. Something that most web
> servers support (for instance Apache Httpd and .htaccess files)
>
> Existing Example:
>
> archives.apache.org represents content from www.apache.org/dist that
> has been "decommissioned" from the mirroring process, of course
> mirrors may maintain copies of these files by not deleting contents.
>
> Ideally, such a mechanism could even be automated based on historical
> download information on the resource. I.E. if the resource hasn't been
> downloaded in 5 years, move it into an archive and provide a redirect
> or notice.
>
> -Mark
>
> Nick Chalko wrote:
>
>> Patrick Chanezon wrote:
>>
>>> Did you specify a lifecycle for artifacts, with some durations, and
>>> a process to decommision them ?
>>>
>>
>> Good question. This may be something to put to the board. My general
>> thought are. "Released" version should live forever, unless a
>> security or other fatal flaw is found in a release.
>>
>> As a minimum I think the latest version of each point release should
>> be kept ie 1.2.x.
>> R,
>> Nick
>>
>
Re: cool uris don't change, don't they ?
Posted by "Mark R. Diggory" <md...@latte.harvard.edu>.
With the amount of versioning going on, eventually a release falls into
a state of non-usage, I suspect there should be room for such a
mechanism, otherwise mirrors will become bloated with unused, outdated,
antiquated and obsolete content.
I suspect some sort of "redirect" mechanism would be sufficient in cases
where an unmirrored archive is used. Something that most web servers
support (for instance Apache Httpd and .htaccess files)
Existing Example:
archives.apache.org represents content from www.apache.org/dist that has
been "decommissioned" from the mirroring process, of course mirrors may
maintain copies of these files by not deleting contents.
Ideally, such a mechanism could even be automated based on historical
download information on the resource. I.E. if the resource hasn't been
downloaded in 5 years, move it into an archive and provide a redirect or
notice.
-Mark
Nick Chalko wrote:
> Patrick Chanezon wrote:
>
>> Did you specify a lifecycle for artifacts, with some durations, and a
>> process to decommision them ?
>>
>
> Good question. This may be something to put to the board. My general
> thought are. "Released" version should live forever, unless a security
> or other fatal flaw is found in a release.
>
> As a minimum I think the latest version of each point release should be
> kept ie 1.2.x.
> R,
> Nick
>
--
Mark Diggory
Software Developer
Harvard MIT Data Center
http://www.hmdc.harvard.edu
Re: cool uris don't change, don't they ?
Posted by Nick Chalko <ni...@chalko.com>.
Patrick Chanezon wrote:
> Did you specify a lifecycle for artifacts, with some durations, and a
> process to decommision them ?
>
Good question. This may be something to put to the board.
My general thought are.
"Released" version should live forever, unless a security or other fatal
flaw is found in a release.
As a minimum I think the latest version of each point release should be
kept ie 1.2.x.
R,
Nick
Re: cool uris don't change, don't they ?
Posted by Jason van Zyl <jv...@maven.org>.
On Tue, 2004-02-03 at 08:46, Jeffrey D. Brekke wrote:
> >>>>> On Tue, 03 Feb 2004 01:21:01 -0500, Jason van Zyl <jv...@maven.org> said:
> [SNIPPED]
>
> >> I'm also wary about the proliferation of shitty snapshots
> >> everywhere.
>
> > Eventually as practices become standard there will be less cruft but
> > the cruft will stay there for all to see.
>
> >> This is less a problem in ibiblio at the moment as uploads are
> >> manual and hence limited. Once an automated way of pushing
> >> snapshots into the repository becomes widely spread, then we could
> >> have an explosion of snapshots that really don't have any value (I
> >> realise that some snapshots do have value, but for the most part
> >> they probably wouldn't have long term value) Flagging artifacts as
> >> transient and then warning about them in build tools might be a
> >> general solution to this.
>
> > Possibly but don't really know what people have used along the way
> > for releases. At some point in the future we may have something that
> > could determine with relative certainty that an artifact was not
> > part of any release this will be a some time in coming and until
> > then, given ibiblios's resources, that there's no downside in
> > keeping the artifacts around except for the rare cases where
> > concerns of security override those of archiving for historical
> > intactness.
>
> [SNIPPED]
>
> I always took the idea of using a SNAPSHOT of anything as a temporary
> dependency that could be removed once a release is put forth. In
> otherwords, use SNAPSHOT's, but don't release with them.
Right, that's the symlinked SNAPSHOT which is attached to a real
timestamped artifact. Those are the things that won't be removed, I
wasn't clear.
> For releases, even alpha/beta/rc, they should remain. I guess I
> didn't know about leaving all SNAPSHOT's around though.
The actual foo-SNAPSHOT.jar gets relinked all the time, it's the actual
timestamped version that it refers to that doesn't get erased.
> I guess we're
> at the mercy of projects that never release anything but SNAPSHOT's in
> which case, any artifact that is put in the repo is *released*.
When the SNAPSHOT is released, it is really a timestamped version. Just
releasing foo-SNAPSHOT.jar is evil, bad and will have to be prevented
using tools if that's what's happening. Using the plugins to release a
snapshot is made by created a timestamped version of the JAR and then on
the server a symlink (or on windows a copy) is created between the real
artifact and the SNAPSHOT. Without deploying a SNAPSHOT the correct way
there is no mapping between the two which means the release plugin
cannot resolve SNAPSHOT names in your POM to real timestamped versions
of the artifacts.
--
jvz.
Jason van Zyl
jason@maven.org
http://maven.apache.org
happiness is like a butterfly: the more you chase it, the more it will
elude you, but if you turn your attention to other things, it will come
and sit softly on your shoulder ...
-- Thoreau
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: cool uris don't change, don't they ?
Posted by "Jeffrey D. Brekke" <jb...@wi.rr.com>.
>>>>> On Tue, 03 Feb 2004 01:21:01 -0500, Jason van Zyl <jv...@maven.org> said:
[SNIPPED]
>> I'm also wary about the proliferation of shitty snapshots
>> everywhere.
> Eventually as practices become standard there will be less cruft but
> the cruft will stay there for all to see.
>> This is less a problem in ibiblio at the moment as uploads are
>> manual and hence limited. Once an automated way of pushing
>> snapshots into the repository becomes widely spread, then we could
>> have an explosion of snapshots that really don't have any value (I
>> realise that some snapshots do have value, but for the most part
>> they probably wouldn't have long term value) Flagging artifacts as
>> transient and then warning about them in build tools might be a
>> general solution to this.
> Possibly but don't really know what people have used along the way
> for releases. At some point in the future we may have something that
> could determine with relative certainty that an artifact was not
> part of any release this will be a some time in coming and until
> then, given ibiblios's resources, that there's no downside in
> keeping the artifacts around except for the rare cases where
> concerns of security override those of archiving for historical
> intactness.
[SNIPPED]
I always took the idea of using a SNAPSHOT of anything as a temporary
dependency that could be removed once a release is put forth. In
otherwords, use SNAPSHOT's, but don't release with them.
For releases, even alpha/beta/rc, they should remain. I guess I
didn't know about leaving all SNAPSHOT's around though. I guess we're
at the mercy of projects that never release anything but SNAPSHOT's in
which case, any artifact that is put in the repo is *released*.
--
=====================================================================
Jeffrey D. Brekke jbrekke@wi.rr.com
Wisconsin, USA brekke@apache.org
ekkerbj@yahoo.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: cool uris don't change, don't they ?
Posted by Jason van Zyl <jv...@maven.org>.
On Tue, 2004-02-03 at 01:12, Ben Walding wrote:
> Jason wrote:
>
> >They are never to be removed from the central repository.
> >
> >
> >
> There are probably a couple of reasons that an artifact would be removed
> / replaced in the repository
>
> 1) Legal reasons, eg. we aren't allowed to distribute them (javamail,
> ejb, ... damned Sun)
This there is just nothing we can do about.
> 2) Corrupt archives - no point in having something in the repository if
> it is corrupt - fix it or remove it
Leave it, and the project does it again.
> 3) Trojanned archives - while I hope this never happens, we would fix /
> remove archives that had been trojanned. However if we can't work out
> how it got trojanned, then that represents a whole other kettle of fish
> in terms of validating the repository.
Sure, but this would certainly be an exceptional case.
>
> Of course there is also some edge cases such as archives that have
> security vulnerabilities. This could be handled by artifact metadata
> that warned clients that the archive had issues.
Have any examples?
> I'm also wary about the proliferation of shitty snapshots everywhere.
Eventually as practices become standard there will be less cruft but the
cruft will stay there for all to see.
> This is less a problem in ibiblio at the moment as uploads are manual
> and hence limited. Once an automated way of pushing snapshots into the
> repository becomes widely spread, then we could have an explosion of
> snapshots that really don't have any value (I realise that some
> snapshots do have value, but for the most part they probably wouldn't
> have long term value) Flagging artifacts as transient and then warning
> about them in build tools might be a general solution to this.
Possibly but don't really know what people have used along the way for
releases. At some point in the future we may have something that could
determine with relative certainty that an artifact was not part of any
release this will be a some time in coming and until then, given
ibiblios's resources, that there's no downside in keeping the artifacts
around except for the rare cases where concerns of security override
those of archiving for historical intactness.
> sd
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
--
jvz.
Jason van Zyl
jason@maven.org
http://maven.apache.org
happiness is like a butterfly: the more you chase it, the more it will
elude you, but if you turn your attention to other things, it will come
and sit softly on your shoulder ...
-- Thoreau
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: cool uris don't change, don't they ?
Posted by Ben Walding <be...@walding.com>.
Jason wrote:
>They are never to be removed from the central repository.
>
>
>
There are probably a couple of reasons that an artifact would be removed
/ replaced in the repository
1) Legal reasons, eg. we aren't allowed to distribute them (javamail,
ejb, ... damned Sun)
2) Corrupt archives - no point in having something in the repository if
it is corrupt - fix it or remove it
3) Trojanned archives - while I hope this never happens, we would fix /
remove archives that had been trojanned. However if we can't work out
how it got trojanned, then that represents a whole other kettle of fish
in terms of validating the repository.
Of course there is also some edge cases such as archives that have
security vulnerabilities. This could be handled by artifact metadata
that warned clients that the archive had issues.
I'm also wary about the proliferation of shitty snapshots everywhere.
This is less a problem in ibiblio at the moment as uploads are manual
and hence limited. Once an automated way of pushing snapshots into the
repository becomes widely spread, then we could have an explosion of
snapshots that really don't have any value (I realise that some
snapshots do have value, but for the most part they probably wouldn't
have long term value) Flagging artifacts as transient and then warning
about them in build tools might be a general solution to this.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: cool uris don't change, don't they ?
Posted by Jason van Zyl <jv...@maven.org>.
On Mon, 2004-02-02 at 22:11, Patrick Chanezon wrote:
> Hi all,
> I've been lurking in that list (repository) for a while, because the
> topic of repositories is of interest to me.
> I think that maven will have a big effect on java development and I
> understand that the apache repository will be where maven stores its
> jars (not sure about that but that would make sense, cross posting to
> maven-dev).
The apache repository is not where Maven stores its JARs. The maven
repository at Apache exists because it was mandated by the board that
artifacts created by Apache projects be distributed from Apache. The
repository@apache doesn't really have anything to do with Maven.
> I just wanted to check if you quantified or considered the requirement
> about how long you would keep artifacts at their url.
In the central repository at Ibiblio they will stay there forever. The
central repository is to be a comprehensive, complete and historical
repository. Things go in and are not removed regardless of any error on
the part of a project submitting an artifact.
> Today I experienced some broken build with an unofficial makefile that
> helps build a system called Syncato (details at
> http://www.chanezon.com/pat/weblog/archives/000131.html).
> Rick Bradley's syncatomatic is a hack, and it broke because it made
> assumptions about xerces 2.3 source distribution to be hosted on an
> apache mirror.
Unlike the Apache maven repository, the ibiblio repository won't ever
have anything culled. So the moral of the story is use the ibiblio
repository.
> I thought: no big deal for a quick hack like this, designed to help
> people out temporarily.
>
> But when most java software gets built using maven, I guess you don't
> want that kind of things to happen. And it could happen all over the
> place, in a few years, if old version of artifacts get decommissionned
> wihtout warning.
No, they won't. Not on Ibiblio at least. There mandate is to help
disseminate OSS software and archive it indefinitely for posterity.
That's why I originally chose to use Ibiblio over using Apache resources
which are limited in comparison to Ibiblio's.
> Did you specify a lifecycle for artifacts, with some durations, and a
> process to decommision them ?
They are never to be removed from the central repository.
--
jvz.
Jason van Zyl
jason@maven.org
http://maven.apache.org
happiness is like a butterfly: the more you chase it, the more it will
elude you, but if you turn your attention to other things, it will come
and sit softly on your shoulder ...
-- Thoreau
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org