You are viewing a plain text version of this content. The canonical link for it is here.
Posted to repository@apache.org by Henri Yandell <fl...@gmail.com> on 2006/04/04 22:56:25 UTC
sources + javadocs
It's all the rage to deploy sources and javadocs to your repos nowadays.
Anyone mind if I go ahead and see how much I can script in the current
repos such that the sources and javadocs are pulled out of the .zip
files in the archive and sucked over into the maven repo?
Hen
Re: sources + javadocs
Posted by Jason van Zyl <ja...@maven.org>.
Jorg Heymans wrote:
> Henri Yandell wrote:
>
>> Anyone mind if I go ahead and see how much I can script in the current
>> repos such that the sources and javadocs are pulled out of the .zip
>> files in the archive and sucked over into the maven repo?
>
> You mean the current maven2 repo? I didn't realize sources and javadoc
> were being packed together in the archive, but if they are by all means
> go for it.
>
> A better strategy long term is to ask all our m2 based projects to
> deploy the sources and javadocs alongside the binaries. Or perhaps this
> can be added as a build plugin that is always executed from asf/pom.xml?
If you release using the release plugin then the sources and javadocs
are always deployed along with the official release.
>
> Jorg
>
>
--
jvz.
Jason van Zyl
jason at maven.org
http://maven.apache.org
the course of true love never did run smooth ...
-- Shakespeare
Re: sources + javadocs
Posted by Jorg Heymans <jh...@apache.org>.
Henri Yandell wrote:
> Anyone mind if I go ahead and see how much I can script in the current
> repos such that the sources and javadocs are pulled out of the .zip
> files in the archive and sucked over into the maven repo?
You mean the current maven2 repo? I didn't realize sources and javadoc
were being packed together in the archive, but if they are by all means
go for it.
A better strategy long term is to ask all our m2 based projects to
deploy the sources and javadocs alongside the binaries. Or perhaps this
can be added as a build plugin that is always executed from asf/pom.xml?
Jorg
Re: sources + javadocs
Posted by "Henk P. Penning" <he...@cs.uu.nl>.
On Tue, 4 Apr 2006, Henri Yandell wrote:
> Date: Tue, 4 Apr 2006 16:01:47 -0700
> From: Henri Yandell <fl...@gmail.com>
> To: repository@apache.org
> Subject: Re: sources + javadocs
>
> On 4/4/06, Henk P. Penning <he...@cs.uu.nl> wrote:
> > Writing scripts to install stuff is easy ;
> > writing stuff that cleans up is not so easy.
> > First solve the not-so-easy part, before you solve the easy part.
>
> I thought the solution was supposed to be easy. Move it out of dist.
Hen,
Ok, providing the stuff is signed.
> > The idea for maven-repository was to have cleanup-hooks :
> > a pointer from the artifact to the source in 'dist/' ;
> > when the source goes out, the artifact can be removed.
> > I haven't seen that idea gone into practice yet, despite
> > good intentions and promises.
>
> Does the pointer even need to remain? The chief problem is that the
> ibiblio sync would delete it (I thought). So would need to stop the
> --delete if it's happening (or whatever the flag is).
?? ; there is no delete on ibiblio
> > Take a look at java-repository ; what percentage of the stuff
> > there is obsolete (given its also in archive.apache.org/dist) ?
>
> Two problems with this question:
>
> 1) Nothing in there is obsolete. It has a different use case than the
> rest of dist/.
Hm, perhaps, up to a point ; there is lots of stuff that has been
obsoleted by newer versions, many times over.
Once upon a time it was decided to split stuff three ways:
-- current production stuff sits in dist (and goes to mirrors)
-- old stuff sits in archive
-- new not-yet-production stuff sits in cvs
Because the repositories sit in dist, contents is limited to
'current production stuff'.
> 2) Nothing in there is in archive.apache.org/dist (afaik). One has jar
> files, the other tends to be zips/tar.gzs.
No ; look at
http://archive.apache.org/dist/java-repository/
it's all there, rsynced from www.apache.org/dist without deletes.
> What's the cause of the need to move to archive - mirroring or disk
> location? If just mirroring, can we just filter out those directories
> from being mirrored? (no idea if Maven/Ant would want this, but I
> don't know of any current reason to mirror them).
The archive.apache.org was invented to make it easier for people
to throw stuff away in dist (don't think too much about it, there's
always a copy on file). Keeping dist clean and lean is good for
-- average downloaders browsing the ASF tree
-- space and bandwidth requirements for mirrors
-- ASF's bandwidth requirements ; countless clients downloading
the entire 'dist'.
I wasn't here when 'java-repository' was invented, and I don't
know why it was a good idea to put in 'dist'.
Another idea : I notice that most 'dist/TLP/' trees are kept clean.
The PMC's don't pay much attention to what in the repositories.
Maybe the PMC's would look after 'dist/TLP/repository' trees.
We could easily (?) split up the repostory tree, bringing them
under the PMC's watchfull eyes. A script to hardlink or softlink
a complete repository tree would be easy.
> Hen
HPP
---------------------------------------------------------------- _
Henk P. Penning, Computer Systems Group R Uithof CGN-A232 _/ \_
Dept of Computer Science, Utrecht University T +31 30 253 4106 / \_/ \
Padualaan 14, 3584CH Utrecht, the Netherlands F +31 30 251 3791 \_/ \_/
http://www.cs.uu.nl/~henkp/ M penning@cs.uu.nl \_/
Re: sources + javadocs
Posted by Henri Yandell <fl...@gmail.com>.
On 4/4/06, Henk P. Penning <he...@cs.uu.nl> wrote:
> On Tue, 4 Apr 2006, Henri Yandell wrote:
>
> > Date: Tue, 4 Apr 2006 14:58:11 -0700
> > From: Henri Yandell <fl...@gmail.com>
> > To: repository@apache.org
> > Subject: Re: sources + javadocs
> >
> > On 4/4/06, Henk P. Penning <he...@cs.uu.nl> wrote:
> > > On Tue, 4 Apr 2006, Henri Yandell wrote:
> > >
> > > > Date: Tue, 4 Apr 2006 13:56:25 -0700
> > > > From: Henri Yandell <fl...@gmail.com>
> > > > To: repository@apache.org
> > > > Subject: sources + javadocs
> > > >
> > > > It's all the rage to deploy sources and javadocs to your repos nowadays.
> > > >
> > > > Anyone mind if I go ahead and see how much I can script in the current
> > > > repos such that the sources and javadocs are pulled out of the .zip
> > > > files in the archive and sucked over into the maven repo?
> > >
> > > Hen,
> > >
> > > The rules in maven-repository/README.txt :
> > >
> > > - only deploy releases voted on by the PMC
> >
> > Out of interest, are we meant to remove ones that weren't voted on?
>
> I don't know too much about this rule. It is a general principle
> for the ASF regarding '/dist/'.
>
> > > - sign all artifacts and POMs (currently, this is a manual step)
> > > - never deploy snapshots or development releases to this repository
> > > - ensure that if a release is deleted from /dist/, it is removed from
> > > here completely also (excluding archiving)
> >
> > Which would be a bad thing from a user point of view (I assume the
> > rsync is deleting at ibiblio if we delete).
>
> No. IIRC ibiblio offered a mirror under condition they wouldn't
> have to delete.
Right, thought that was something they were hot on.
> Please take this point (automatic cleanup) very serious.
>
> The java-repository is a big mess and this shouldn't be repeated
> in the maven-repository. Writing scripts to install stuff is easy ;
> writing stuff that cleans up is not so easy.
>
> First solve the not-so-easy part, before you solve the easy part.
I thought the solution was supposed to be easy. Move it out of dist.
> The idea for maven-repository was to have cleanup-hooks :
> a pointer from the artifact to the source in 'dist/' ;
> when the source goes out, the artifact can be removed.
> I haven't seen that idea gone into practice yet, despite
> good intentions and promises.
Does the pointer even need to remain? The chief problem is that the
ibiblio sync would delete it (I thought). So would need to stop the
--delete if it's happening (or whatever the flag is).
> Take a look at java-repository ; what percentage of the stuff
> there is obsolete (given its also in archive.apache.org/dist) ?
Two problems with this question:
1) Nothing in there is obsolete. It has a different use case than the
rest of dist/.
2) Nothing in there is in archive.apache.org/dist (afaik). One has jar
files, the other tends to be zips/tar.gzs.
What's the cause of the need to move to archive - mirroring or disk
location? If just mirroring, can we just filter out those directories
from being mirrored? (no idea if Maven/Ant would want this, but I
don't know of any current reason to mirror them).
Hen
Re: sources + javadocs
Posted by "Henk P. Penning" <he...@cs.uu.nl>.
On Tue, 4 Apr 2006, Henri Yandell wrote:
> Date: Tue, 4 Apr 2006 14:58:11 -0700
> From: Henri Yandell <fl...@gmail.com>
> To: repository@apache.org
> Subject: Re: sources + javadocs
>
> On 4/4/06, Henk P. Penning <he...@cs.uu.nl> wrote:
> > On Tue, 4 Apr 2006, Henri Yandell wrote:
> >
> > > Date: Tue, 4 Apr 2006 13:56:25 -0700
> > > From: Henri Yandell <fl...@gmail.com>
> > > To: repository@apache.org
> > > Subject: sources + javadocs
> > >
> > > It's all the rage to deploy sources and javadocs to your repos nowadays.
> > >
> > > Anyone mind if I go ahead and see how much I can script in the current
> > > repos such that the sources and javadocs are pulled out of the .zip
> > > files in the archive and sucked over into the maven repo?
> >
> > Hen,
> >
> > The rules in maven-repository/README.txt :
> >
> > - only deploy releases voted on by the PMC
>
> Out of interest, are we meant to remove ones that weren't voted on?
I don't know too much about this rule. It is a general principle
for the ASF regarding '/dist/'.
> > - sign all artifacts and POMs (currently, this is a manual step)
> > - never deploy snapshots or development releases to this repository
> > - ensure that if a release is deleted from /dist/, it is removed from
> > here completely also (excluding archiving)
>
> Which would be a bad thing from a user point of view (I assume the
> rsync is deleting at ibiblio if we delete).
No. IIRC ibiblio offered a mirror under condition they wouldn't
have to delete.
Please take this point (automatic cleanup) very serious.
The java-repository is a big mess and this shouldn't be repeated
in the maven-repository. Writing scripts to install stuff is easy ;
writing stuff that cleans up is not so easy.
First solve the not-so-easy part, before you solve the easy part.
The idea for maven-repository was to have cleanup-hooks :
a pointer from the artifact to the source in 'dist/' ;
when the source goes out, the artifact can be removed.
I haven't seen that idea gone into practice yet, despite
good intentions and promises.
Take a look at java-repository ; what percentage of the stuff
there is obsolete (given its also in archive.apache.org/dist) ?
> Hen
HPP
---------------------------------------------------------------- _
Henk P. Penning, Computer Systems Group R Uithof CGN-A232 _/ \_
Dept of Computer Science, Utrecht University T +31 30 253 4106 / \_/ \
Padualaan 14, 3584CH Utrecht, the Netherlands F +31 30 251 3791 \_/ \_/
http://www.cs.uu.nl/~henkp/ M penning@cs.uu.nl \_/
Re: sources + javadocs
Posted by Henri Yandell <fl...@gmail.com>.
On 4/4/06, Henk P. Penning <he...@cs.uu.nl> wrote:
> On Tue, 4 Apr 2006, Henri Yandell wrote:
>
> > Date: Tue, 4 Apr 2006 13:56:25 -0700
> > From: Henri Yandell <fl...@gmail.com>
> > To: repository@apache.org
> > Subject: sources + javadocs
> >
> > It's all the rage to deploy sources and javadocs to your repos nowadays.
> >
> > Anyone mind if I go ahead and see how much I can script in the current
> > repos such that the sources and javadocs are pulled out of the .zip
> > files in the archive and sucked over into the maven repo?
>
> Hen,
>
> The rules in maven-repository/README.txt :
>
> - only deploy releases voted on by the PMC
Out of interest, are we meant to remove ones that weren't voted on?
> - sign all artifacts and POMs (currently, this is a manual step)
> - never deploy snapshots or development releases to this repository
> - ensure that if a release is deleted from /dist/, it is removed from
> here completely also (excluding archiving)
Which would be a bad thing from a user point of view (I assume the
rsync is deleting at ibiblio if we delete).
> - no artifacts are allowed from projects under incubation
> - make sure you have your maven settings correctly set to make
> directories group writeable.
>
> -- everything you generate from source 'dist/XXX', must be removed
> /automatically/ from the repository if/when 'dist/XXX' is removed
> -- every jar, zip, tar.gz etc must be signed.
Might have to chew a bit of bandwidth while I get the files locally so
I can sign them; otherwise, no prob.
> Infrastructure frowns upon (forbids?) automatic deployment.
> Check with them, and possibly the board.
Will do it for Commons/Jakarta first with a vote. If it seems
scriptable, maybe then try to hit more.
Hen
Re: sources + javadocs
Posted by "Henk P. Penning" <he...@cs.uu.nl>.
On Tue, 4 Apr 2006, Henri Yandell wrote:
> Date: Tue, 4 Apr 2006 13:56:25 -0700
> From: Henri Yandell <fl...@gmail.com>
> To: repository@apache.org
> Subject: sources + javadocs
>
> It's all the rage to deploy sources and javadocs to your repos nowadays.
>
> Anyone mind if I go ahead and see how much I can script in the current
> repos such that the sources and javadocs are pulled out of the .zip
> files in the archive and sucked over into the maven repo?
Hen,
The rules in maven-repository/README.txt :
- only deploy releases voted on by the PMC
- sign all artifacts and POMs (currently, this is a manual step)
- never deploy snapshots or development releases to this repository
- ensure that if a release is deleted from /dist/, it is removed from
here completely also (excluding archiving)
- no artifacts are allowed from projects under incubation
- make sure you have your maven settings correctly set to make
directories group writeable.
-- everything you generate from source 'dist/XXX', must be removed
/automatically/ from the repository if/when 'dist/XXX' is removed
-- every jar, zip, tar.gz etc must be signed.
Infrastructure frowns upon (forbids?) automatic deployment.
Check with them, and possibly the board.
> Hen
HPP
---------------------------------------------------------------- _
Henk P. Penning, Computer Systems Group R Uithof CGN-A232 _/ \_
Dept of Computer Science, Utrecht University T +31 30 253 4106 / \_/ \
Padualaan 14, 3584CH Utrecht, the Netherlands F +31 30 251 3791 \_/ \_/
http://www.cs.uu.nl/~henkp/ M penning@cs.uu.nl \_/