You are viewing a plain text version of this content. The canonical link for it is here.
Posted to repository@apache.org by Stefano Mazzocchi <st...@apache.org> on 2006/05/30 08:15:53 UTC
[Plan of action] Setting up an official maven repository for the
ASF
So, following up on my board@ message, here we should start to work on a
plan of action for setting up an official maven repository for the ASF
that would meet both board's legal concerns and infra's
technical/security concerns and general social feasibility concerns.
I know Brett has been thinking about this a lot so I'll leave the
microphone to him to start.
Let's rock.
--
Stefano.
Re: [Plan of action] Setting up an official maven repository for the ASF
Posted by Dain Sundstrom <da...@iq80.com>.
+1
Great work Henri!
-dain
On Jun 6, 2006, at 11:45 PM, Henri Yandell wrote:
> Nothing from Brett yet, so braindump from me.
>
> Two classes of repository - m1 and m2. We should focus on short-term
> goals for the m1 one and long-term for the m2 one.
>
> Some goals:
>
> * Development ease. It should not take forever to get a dependency
> deployed.
> * Authentic. PGP .asc files for each release.
> * Official. Projects should know their code is being placed in the
> repository.
> * Oversight. People shouldn't be deploying jars randomly.
> * Complete. Javadoc, Source and Binary jars.
> * Stable. Files should not be easy to change.
>
> Multiple repositories are needed:
>
> * ASF Release Repository.
> * ASF Snapshot Repository. Deployed to from CI.
> * ASF Development Repository. May contain 3rd party jars and manual
> snapshots.
>
> Some random ideas:
>
> * Monitoring. Have a script that mails this list when something is
> added to the release or development repositories. Much like a cvs
> commit in another project.
>
> * Tighten things up in a staggered way. Every couple of months we can
> enforce something new. ie) All non-PGP signed jars will be removed by
> X.
>
> * Pull the Maven repositories out of the mirroring system. It's
> unnecessary. With the rsync being down atm, there's not a lot of
> reason to avoid it. In fact, rm the ones in the maven repos and move
> the ones in archive.apache to repo.apache.org. Need various
> repo.apache.org directory names.
>
> Hen
>
> On 5/29/06, Stefano Mazzocchi <st...@apache.org> wrote:
>> So, following up on my board@ message, here we should start to
>> work on a
>> plan of action for setting up an official maven repository for the
>> ASF
>> that would meet both board's legal concerns and infra's
>> technical/security concerns and general social feasibility concerns.
>>
>> I know Brett has been thinking about this a lot so I'll leave the
>> microphone to him to start.
>>
>> Let's rock.
>>
>> --
>> Stefano.
>>
>>
Re: [Plan of action] Setting up an official maven repository for the ASF
Posted by Brett Porter <br...@gmail.com>.
Thanks Hen, good start. I somehow missed Stefano's message.
http://maven.apache.org/repository-manager/ has tools that should
cover the majority of what you have here so far. It's getting much
closer to being functional enough to use.
We should move to use WebDAV for deployment.
But I'll dig up those requirements before I jump at the rest of the details.
- Brett
On 07/06/06, Henri Yandell <fl...@gmail.com> wrote:
> Nothing from Brett yet, so braindump from me.
>
> Two classes of repository - m1 and m2. We should focus on short-term
> goals for the m1 one and long-term for the m2 one.
>
> Some goals:
>
> * Development ease. It should not take forever to get a dependency deployed.
> * Authentic. PGP .asc files for each release.
> * Official. Projects should know their code is being placed in the repository.
> * Oversight. People shouldn't be deploying jars randomly.
> * Complete. Javadoc, Source and Binary jars.
> * Stable. Files should not be easy to change.
>
> Multiple repositories are needed:
>
> * ASF Release Repository.
> * ASF Snapshot Repository. Deployed to from CI.
> * ASF Development Repository. May contain 3rd party jars and manual snapshots.
>
> Some random ideas:
>
> * Monitoring. Have a script that mails this list when something is
> added to the release or development repositories. Much like a cvs
> commit in another project.
>
> * Tighten things up in a staggered way. Every couple of months we can
> enforce something new. ie) All non-PGP signed jars will be removed by
> X.
>
> * Pull the Maven repositories out of the mirroring system. It's
> unnecessary. With the rsync being down atm, there's not a lot of
> reason to avoid it. In fact, rm the ones in the maven repos and move
> the ones in archive.apache to repo.apache.org. Need various
> repo.apache.org directory names.
>
> Hen
>
> On 5/29/06, Stefano Mazzocchi <st...@apache.org> wrote:
> > So, following up on my board@ message, here we should start to work on a
> > plan of action for setting up an official maven repository for the ASF
> > that would meet both board's legal concerns and infra's
> > technical/security concerns and general social feasibility concerns.
> >
> > I know Brett has been thinking about this a lot so I'll leave the
> > microphone to him to start.
> >
> > Let's rock.
> >
> > --
> > Stefano.
> >
> >
>
--
Apache Maven - http://maven.apache.org
"Better Builds with Maven" book - http://library.mergere.com/
Re: [Plan of action] Setting up an official maven repository for the ASF
Posted by Henri Yandell <fl...@gmail.com>.
On 6/7/06, Stefano Mazzocchi <st...@apache.org> wrote:
> Henri Yandell wrote:
> > * Complete. Javadoc, Source and Binary jars.
>
> yup... I also believe that the maven2 plugin for eclipse is making
> people aware of why this is needed.
>
> Would also be nice if we could automate the extraction of such javadoc
> jars and provide a javadocs.apache.org thingy
Way ahead of you :)
http://people.apache.org/~bayard/multidoc-jnr/
http://dist.osjava.org/releases/multidoc-jnr/
The osjava one has all new pretty graphs.
It's a mess of Java/Ruby/Bash at the moment, but runs on a maven repository.
Hen
Re: [Plan of action] Setting up an official maven repository for the ASF
Posted by Niclas Hedhman <ni...@hedhman.org>.
On Friday 09 June 2006 04:43, Stefano Mazzocchi wrote:
> The latest trend in term of content distribution is "magnet" URI scheme
> which a very simple yet radically clever new way of using the web.
>
> http://en.wikipedia.org/wiki/Magnet:_URI_scheme
>
> Instead of a URL such as the above, you get a magnet URI such as
>
> magnet:?xt=urn:sha1:YNCKHTQCWBTRNJIV4WNAE52SJUQCZO5C
Really cool, although it still only solves part of the total issue. Still need
representation of meta-data, unless you are going to dictate that everyone
MUST use Maven POMs.
For instance, how do find the licensing of the above? Or how do I make the
licensing information available, if the original authors are not interested?
That's why *I* think RDF comes in handy, but I agree that the XML
serialization sux big time and tool support is weak.
Cheers
Niclas
Anecdote; The Jini project allows for temporary URLs based on MD5 checksums,
using a custom protocol httpmd, such as httpmd://89d245a488d96ae2, mainly
used for dynamic codebases in Jini federations.
However, it requires the presence of Jini infrastructure, and not suitable for
global scale.
Re: [Plan of action] Setting up an official maven repository for the ASF
Posted by Niclas Hedhman <ni...@hedhman.org>.
On Friday 09 June 2006 04:43, Stefano Mazzocchi wrote:
> So, imagine that your maven was an azureus DHT client and that your POM
> contains a bunch of magnet URIs for the dependency jars... then what
> happens is:
Too bad that Azureus is GPL and Maven can't incorporate code in its core...
Cheers
Niclas
Re: [Plan of action] Setting up an official maven repository for
the ASF
Posted by Stefano Mazzocchi <st...@apache.org>.
Niclas Hedhman wrote:
> On Thursday 08 June 2006 08:07, Alex Karasulu wrote:
>
>> Niclas Hedhman had some very interesting ideas on using RDF with a build
>> system which did not centralize a repository there by preventing many of
>> the bottle neck issues we have today with central repos like ibiblio.
>>
>> Perhaps Niclas can elaborate a bit on this ingenious idea.
>
> The principle was about applying RDF meta on top of artifacts, including
> authentication and other meta information, available from RDF search engines
> in the distributed fashion RDF is designed to work. Design points were;
>
> * No centralized repository of everything.
> * Distributed search engines.
> * Working with existing, published artifacts without participation of the
> publishers, such as sourceforge.net projects.
> * Ensured Authenticity.
>
> Sure Stefano can provide the details in how to best go about doing this.
>
> Unfortunately, before getting to this stage, I hanged in the towel due to lack
> of time.
>
>> BTW the p2p propagation of artifacts via bittorent is a great idea.
>
> Yep. I like that idea a lot as well. Having the users participate in the P2P
> network is not very hard. Maven doesn't need to be involved, and a pure
> voluntary effort of starting a separate daemon would probably give enough
> peers to manage. After all, we are not talking multi GB downloads (yet) ;o)
> However, How well does P2P perform when the files are small? Would the
> establishment of connection take too long, and we end up with very slow
> downloads?
RDF is a graph data model with a weird XML serialization and there are
no standards or tools that would help us in achieving the above goals,
so I think it is wrong to mention what data model to use, especially
since maven is not RDF aware and honestly wouldn't gain much if it was.
- o -
The latest trend in term of content distribution is "magnet" URI scheme
which a very simple yet radically clever new way of using the web.
http://en.wikipedia.org/wiki/Magnet:_URI_scheme
Instead of a URL such as the above, you get a magnet URI such as
magnet:?xt=urn:sha1:YNCKHTQCWBTRNJIV4WNAE52SJUQCZO5C
and this is basically a massive hash into a globally interconnected
hashtable (google up DHT, if you want to know more).
Note how:
1) there is no protocol description there.. the client will connect to
all the networks it knows and try to find a reference for that hash. You
can find one, zero or more than one. The 160 bits of address space
guarantee some low probability of collision. Zero means that your client
might not know how to access the network that content is located on.
2) there is no need for a DNS anymore! in an HTTP URI which is used as
a URL, DNS provides the first host->IP translation and the HTTP server
provides the path -> bitstream translation. Here the two are merged into
one... which means that you can publish content even without having a
registered domain! as long as you know how to hook up your content to an
existing transport network and hand off the magnet URI to somebody.
Modern bittorrent clients don't need trackers anymore. (trackers are the
equivalent of DNSs for bittorrent, allowing you to ask for what IP
addresses currently contain that file you are looking for). Azureus, for
example, can restore your torrent even if the tracker is no longer
functioning, using a distributed hashtable to store the SHA1->IP
information.
So, imagine that your maven was an azureus DHT client and that your POM
contains a bunch of magnet URIs for the dependency jars... then what
happens is:
1) your maven asks the azureus DHT about IP addresses that have that
SHA1 hash. [note how there is no central point of failure/control in
this system and it's already running with an average of 800k nodes alive
each day and the code is written in java and open source]
2) then the bittorrent transfer is initiated between you and all the IP
addresses that have those files and are seeding them or currently
downloading them.
Since maven jars are very small, the swarm wouldn't really help that
much: your maven client would download a jar before the swarm could even
percolate the information that you are downloading it.
But your maven client might be left 'seeding' the jars that you have
downloaded... basically making that every user that wants an
instantaneous, transparent and bandwidth-shaped mirror of the jars that
you use. Nice social way to pay back.
The two missing pieces are:
1) trust: each pom/jar should be digitally sign and the signature
should allow the client to infer the dependency of signatures. Trust
implies that you stop at a signature that you trust. These parameters
should, of course, be user configurable.
2) search: the azureus DHT is not searcheable because it only contains
a hash->ip information and you cannot reconstruct the hash of the POM
from the hash of the jar. Hashes are, by design, opaque, so it's
impossible to scan the azureus DHT for maven packages, get their poms
and provide search on it.
It is fair to note that there is nothing in the bittorrent architecture
that implies trust or search. This is a design decision, and IMO, a good
one.
--
Stefano.
Re: [Plan of action] Setting up an official maven repository for the ASF
Posted by Brett Porter <br...@gmail.com>.
On 08/06/06, Niclas Hedhman <ni...@hedhman.org> wrote:
> Yep. I like that idea a lot as well. Having the users participate in the P2P
> network is not very hard. Maven doesn't need to be involved, and a pure
> voluntary effort of starting a separate daemon would probably give enough
> peers to manage. After all, we are not talking multi GB downloads (yet) ;o)
> However, How well does P2P perform when the files are small? Would the
> establishment of connection take too long, and we end up with very slow
> downloads?
That has been one of my concerns with it.
The others are:
- a large portion of Maven users in corporate settings are probably
not in a position to the share (or use it if it is not http traffic) -
though it seems like there are a couple of alternatives for this
- you better clearly separate your shareable repository from your
general repository or your corporate code starts finding it's way out
of the building.
- Brett
--
Apache Maven - http://maven.apache.org
"Better Builds with Maven" book - http://library.mergere.com/
Re: [Plan of action] Setting up an official maven repository for the ASF
Posted by Niclas Hedhman <ni...@hedhman.org>.
On Thursday 08 June 2006 08:07, Alex Karasulu wrote:
> Niclas Hedhman had some very interesting ideas on using RDF with a build
> system which did not centralize a repository there by preventing many of
> the bottle neck issues we have today with central repos like ibiblio.
>
> Perhaps Niclas can elaborate a bit on this ingenious idea.
The principle was about applying RDF meta on top of artifacts, including
authentication and other meta information, available from RDF search engines
in the distributed fashion RDF is designed to work. Design points were;
* No centralized repository of everything.
* Distributed search engines.
* Working with existing, published artifacts without participation of the
publishers, such as sourceforge.net projects.
* Ensured Authenticity.
Sure Stefano can provide the details in how to best go about doing this.
Unfortunately, before getting to this stage, I hanged in the towel due to lack
of time.
> BTW the p2p propagation of artifacts via bittorent is a great idea.
Yep. I like that idea a lot as well. Having the users participate in the P2P
network is not very hard. Maven doesn't need to be involved, and a pure
voluntary effort of starting a separate daemon would probably give enough
peers to manage. After all, we are not talking multi GB downloads (yet) ;o)
However, How well does P2P perform when the files are small? Would the
establishment of connection take too long, and we end up with very slow
downloads?
Cheers
Niclas
Re: [Plan of action] Setting up an official maven repository for
the ASF
Posted by Alex Karasulu <ao...@bellsouth.net>.
Dain Sundstrom wrote:
> On Jun 7, 2006, at 10:05 AM, Stefano Mazzocchi wrote:
>
>> I've had a wild idea the other day, inspired by some talks with some
>> other digital library fellows that were complaining about bandwidth
>> consumption. They pointed me at
>>
>> http://osprey.ibiblio.org/
>>
>> which is something that ibiblio does to allow transparent
>> bittorrent-ization of resources.
>>
>> And I thought... hmmm, we could take Azureus (which is a kickass
>> bittorrent implementation in java), strip down the transport part and
>> integrate it in maven. Then use osprey on repo.apache.org to
>> bittorrent-ize things.
>>
>> Thoughts?
>
>
> I love this idea. This came up at some Java conference a while back,
> but none of us knew of a Java bittorrent client....
>
> I think the trick to pulling this off is to have a way to verify
> downloads using a secure signature server of some kind (which you
> mentioned in you email but I choped out :) ). We would also need to
> make sure that there were multiple seeds for the repository or we are
> just a mirror again. It would be nice to be able to have users
> participate in the torrent but that seems unlikely since the maven
> process isn't running that long (except when building geronimo ;) ).
>
> One final thought, we would still need an http repository for those
> behind corporate firewalls, because AFAIK they would get terrible
> connections due to the fairness algorithms in bittorrent.
Niclas Hedhman had some very interesting ideas on using RDF with a build
system which did not centralize a repository there by preventing many of
the bottle neck issues we have today with central repos like ibiblio.
Perhaps Niclas can elaborate a bit on this ingenious idea.
Sounded related although not totally on topic.
BTW the p2p propagation of artifacts via bittorent is a great idea.
Alex
Re: [Plan of action] Setting up an official maven repository for the ASF
Posted by Dain Sundstrom <da...@iq80.com>.
On Jun 7, 2006, at 10:05 AM, Stefano Mazzocchi wrote:
> I've had a wild idea the other day, inspired by some talks with some
> other digital library fellows that were complaining about bandwidth
> consumption. They pointed me at
>
> http://osprey.ibiblio.org/
>
> which is something that ibiblio does to allow transparent
> bittorrent-ization of resources.
>
> And I thought... hmmm, we could take Azureus (which is a kickass
> bittorrent implementation in java), strip down the transport part and
> integrate it in maven. Then use osprey on repo.apache.org to
> bittorrent-ize things.
>
> Thoughts?
I love this idea. This came up at some Java conference a while back,
but none of us knew of a Java bittorrent client....
I think the trick to pulling this off is to have a way to verify
downloads using a secure signature server of some kind (which you
mentioned in you email but I choped out :) ). We would also need to
make sure that there were multiple seeds for the repository or we are
just a mirror again. It would be nice to be able to have users
participate in the torrent but that seems unlikely since the maven
process isn't running that long (except when building geronimo ;) ).
One final thought, we would still need an http repository for those
behind corporate firewalls, because AFAIK they would get terrible
connections due to the fairness algorithms in bittorrent.
-dain
Re: [Plan of action] Setting up an official maven repository for
the ASF
Posted by Geir Magnusson Jr <ge...@apache.org>.
Brett Porter wrote:
> On 08/06/06, Geir Magnusson Jr <ge...@apache.org> wrote:
>> Brett Porter wrote:
>> > The context of what Hen proposes is things required to build Apache
>> > stuff, so it would seem to fit within that definition. "Distribution"
>> > is too loose a term in this regard unfortunately.
>>
>> While the intention is good, some might still consider it "distribution"
>> irrespective of how loose you believe the term is.
>
> Yes, I understand. So, I think we've established there's no legal
> barrier to this - all we have is that some directors have in the past
> said this is not desirable. So we need something from the board that
> says otherwise, and in that distribution should be very clearly
> defined (or the word not used at all :)
>
Right - given all the effort being put into this, this seems like a
small thing to get out of the way once and for all. It would dispel the
confusion....
Maybe Stefano can give the word here...
geir
Re: [Plan of action] Setting up an official maven repository for the ASF
Posted by Brett Porter <br...@gmail.com>.
On 09/06/06, Geir Magnusson Jr <ge...@apache.org> wrote:
> > The ASF development repository is a place to facilitate usage in Maven
> > of things not in either of the above. That is, 3rd party software not
> > in the central repository.
>
> Because ibiblio and others are too slow?
Not really - I was referring to things not on central at all. This
would be developments builds (we don't allow them to be uploaded), or
things that just aren't there yet.
Cheers,
Brett
--
Apache Maven - http://maven.apache.org
"Better Builds with Maven" book - http://library.mergere.com/
Re: [Plan of action] Setting up an official maven repository for
the ASF
Posted by Geir Magnusson Jr <ge...@apache.org>.
Brett Porter wrote:
> On 08/06/06, Geir Magnusson Jr <ge...@apache.org> wrote:
>
>> > Otherwise people building releases will
>> > hit people.apache.org (and there's no reason for it - releases of 3rd
>> > party stuff can be uploaded to ibiblio).
>>
>> I don't understand what you are saying here.
>
> Ok, if I release something, it needs to be reproducible. For that to
> happen, the dependencies need to be both always available and
> unchanging. That's meant to be a contract of the central Maven
> repository (it unfortunately hasn't always been the case), where
> official releases from all over the software world go. For apache
> built things, that would be the contract of the ASF repository, but
> not the other two.
>
> The ASF snapshot repository would house development versions of Apache
> software, the rough equivalent of "nightly builds".
Yep
>
> The ASF development repository is a place to facilitate usage in Maven
> of things not in either of the above. That is, 3rd party software not
> in the central repository.
Because ibiblio and others are too slow?
>
> We don't want everyone building our software having to hit our servers
> if they can use a mirror. That leads to saying the last is only for
> development versions where it is required, but for released software,
> the source should only reference mirror repositories - the only one of
> which at the moment is the maven central repo.
>
> Hope that clears it up.
Yep - thx
geir
>
> - Brett
>
>
Re: [Plan of action] Setting up an official maven repository for the ASF
Posted by Brett Porter <br...@gmail.com>.
On 08/06/06, Geir Magnusson Jr <ge...@apache.org> wrote:
> Brett Porter wrote:
> > The context of what Hen proposes is things required to build Apache
> > stuff, so it would seem to fit within that definition. "Distribution"
> > is too loose a term in this regard unfortunately.
>
> While the intention is good, some might still consider it "distribution"
> irrespective of how loose you believe the term is.
Yes, I understand. So, I think we've established there's no legal
barrier to this - all we have is that some directors have in the past
said this is not desirable. So we need something from the board that
says otherwise, and in that distribution should be very clearly
defined (or the word not used at all :)
This is, in essence, what I thought Stefano was proposing on the board list.
> > At least until we have some sort of mirroring, this must only be for
> > depending on development versions of 3rd party things, and not
> > incorporated into releases.
>
> I don't understand that - these are released versions of 3rd party things.
"these" ? I was talking about what Hen defined it as, which I
understood to be just development versions, which should not be
incorporated into releases.
> > Otherwise people building releases will
> > hit people.apache.org (and there's no reason for it - releases of 3rd
> > party stuff can be uploaded to ibiblio).
>
> I don't understand what you are saying here.
Ok, if I release something, it needs to be reproducible. For that to
happen, the dependencies need to be both always available and
unchanging. That's meant to be a contract of the central Maven
repository (it unfortunately hasn't always been the case), where
official releases from all over the software world go. For apache
built things, that would be the contract of the ASF repository, but
not the other two.
The ASF snapshot repository would house development versions of Apache
software, the rough equivalent of "nightly builds".
The ASF development repository is a place to facilitate usage in Maven
of things not in either of the above. That is, 3rd party software not
in the central repository.
We don't want everyone building our software having to hit our servers
if they can use a mirror. That leads to saying the last is only for
development versions where it is required, but for released software,
the source should only reference mirror repositories - the only one of
which at the moment is the maven central repo.
Hope that clears it up.
- Brett
Re: [Plan of action] Setting up an official maven repository for
the ASF
Posted by Geir Magnusson Jr <ge...@apache.org>.
Brett Porter wrote:
> The context of what Hen proposes is things required to build Apache
> stuff, so it would seem to fit within that definition. "Distribution"
> is too loose a term in this regard unfortunately.
While the intention is good, some might still consider it "distribution"
irrespective of how loose you believe the term is.
>
> At least until we have some sort of mirroring, this must only be for
> depending on development versions of 3rd party things, and not
> incorporated into releases.
I don't understand that - these are released versions of 3rd party things.
> Otherwise people building releases will
> hit people.apache.org (and there's no reason for it - releases of 3rd
> party stuff can be uploaded to ibiblio).
I don't understand what you are saying here.
My only point was that in the past, some have considered making 3rd
party software available to the general public in a system clearly meant
as a general distribution mechanism (as compared to burying in SVN or
CVS as part of a projects /lib directory) to be a form of distribution,
which according to some not something to be done from ASF infrastructure.
This point of view may have changed, so I was just making sure that was
the case.
geir
>
> - Brett
>
> On 08/06/06, Geir Magnusson Jr <ge...@apache.org> wrote:
>> We used to have explicit statements from directors regarding not acting
>> as distributors for non-apache artifacts that are not part of a larger
>> program...
>>
>> That could be a thing of the past now... so are you sure?
>>
>> geir
>>
>>
>> Henri Yandell wrote:
>> > On 6/7/06, Carlos Sanchez <ca...@apache.org> wrote:
>> >
>> >> > > Multiple repositories are needed:
>> >> > >
>> >> > > * ASF Release Repository.
>> >> > > * ASF Snapshot Repository. Deployed to from CI.
>> >> > > * ASF Development Repository. May contain 3rd party jars and
>> manual
>> >> > > snapshots.
>> >> >
>> >> > Can you elaborate more on this?
>> >>
>> >> I thought the repo was only for Apache stuff, so the ASF Development
>> >> Repository doesn't make sense
>> >
>> > No legal or policy reason not to ship non-Apache stuff. Provided it
>> > obeys legal/policy. If our projects need that dependency, we could
>> > have a loose warez-style directory for them to put things in.
>> >
>> > The need has definitely been shown.
>> >
>> > Hen
>> >
>> >
>>
>
>
Re: [Plan of action] Setting up an official maven repository for the ASF
Posted by Brett Porter <br...@gmail.com>.
The context of what Hen proposes is things required to build Apache
stuff, so it would seem to fit within that definition. "Distribution"
is too loose a term in this regard unfortunately.
At least until we have some sort of mirroring, this must only be for
depending on development versions of 3rd party things, and not
incorporated into releases. Otherwise people building releases will
hit people.apache.org (and there's no reason for it - releases of 3rd
party stuff can be uploaded to ibiblio).
- Brett
On 08/06/06, Geir Magnusson Jr <ge...@apache.org> wrote:
> We used to have explicit statements from directors regarding not acting
> as distributors for non-apache artifacts that are not part of a larger
> program...
>
> That could be a thing of the past now... so are you sure?
>
> geir
>
>
> Henri Yandell wrote:
> > On 6/7/06, Carlos Sanchez <ca...@apache.org> wrote:
> >
> >> > > Multiple repositories are needed:
> >> > >
> >> > > * ASF Release Repository.
> >> > > * ASF Snapshot Repository. Deployed to from CI.
> >> > > * ASF Development Repository. May contain 3rd party jars and manual
> >> > > snapshots.
> >> >
> >> > Can you elaborate more on this?
> >>
> >> I thought the repo was only for Apache stuff, so the ASF Development
> >> Repository doesn't make sense
> >
> > No legal or policy reason not to ship non-Apache stuff. Provided it
> > obeys legal/policy. If our projects need that dependency, we could
> > have a loose warez-style directory for them to put things in.
> >
> > The need has definitely been shown.
> >
> > Hen
> >
> >
>
--
Apache Maven - http://maven.apache.org
"Better Builds with Maven" book - http://library.mergere.com/
Re: [Plan of action] Setting up an official maven repository for
the ASF
Posted by Geir Magnusson Jr <ge...@apache.org>.
We used to have explicit statements from directors regarding not acting
as distributors for non-apache artifacts that are not part of a larger
program...
That could be a thing of the past now... so are you sure?
geir
Henri Yandell wrote:
> On 6/7/06, Carlos Sanchez <ca...@apache.org> wrote:
>
>> > > Multiple repositories are needed:
>> > >
>> > > * ASF Release Repository.
>> > > * ASF Snapshot Repository. Deployed to from CI.
>> > > * ASF Development Repository. May contain 3rd party jars and manual
>> > > snapshots.
>> >
>> > Can you elaborate more on this?
>>
>> I thought the repo was only for Apache stuff, so the ASF Development
>> Repository doesn't make sense
>
> No legal or policy reason not to ship non-Apache stuff. Provided it
> obeys legal/policy. If our projects need that dependency, we could
> have a loose warez-style directory for them to put things in.
>
> The need has definitely been shown.
>
> Hen
>
>
Re: [Plan of action] Setting up an official maven repository for the ASF
Posted by Henri Yandell <fl...@gmail.com>.
On 6/7/06, Carlos Sanchez <ca...@apache.org> wrote:
> > > Multiple repositories are needed:
> > >
> > > * ASF Release Repository.
> > > * ASF Snapshot Repository. Deployed to from CI.
> > > * ASF Development Repository. May contain 3rd party jars and manual
> > > snapshots.
> >
> > Can you elaborate more on this?
>
> I thought the repo was only for Apache stuff, so the ASF Development
> Repository doesn't make sense
No legal or policy reason not to ship non-Apache stuff. Provided it
obeys legal/policy. If our projects need that dependency, we could
have a loose warez-style directory for them to put things in.
The need has definitely been shown.
Hen
Re: [Plan of action] Setting up an official maven repository for the ASF
Posted by Carlos Sanchez <ca...@apache.org>.
On 6/7/06, Stefano Mazzocchi <st...@apache.org> wrote:
> Henri Yandell wrote:
> > Nothing from Brett yet, so braindump from me.
>
> Henry, thanks for your reply.
>
> > Two classes of repository - m1 and m2. We should focus on short-term
> > goals for the m1 one and long-term for the m2 one.
>
> Agreed.
>
We can set up regexp rules in httpd for m1 -> m2 url conversion,
that's what we do at ibiblio.
> > Some goals:
> >
> > * Development ease. It should not take forever to get a dependency
> > deployed.
>
> yes, although if shouldn't make it so easy that people will want to do
> it without reading the "readme".
>
> > * Authentic. PGP .asc files for each release.
>
> I've been thinking about this and I suspect that this requires some
> Maven changes. I mean, what's the point of having a PGP file if maven
> doesn't use it? and currently the POM doesn't have the metadata to
> locate the official PGP file (which, remember, should always fetched
> from the original location never from a mirror.
Yep, something to be added in the future
>
> > * Official. Projects should know their code is being placed in the
> > repository.
>
> Yes and I think that the PGP signature should contain trace of who did
> it, so that there is some trace.
>
> > * Oversight. People shouldn't be deploying jars randomly.
>
> if we do the above, we solve this as well: knowing that you can be
> traced, random stuff would slow down.
>
> > * Complete. Javadoc, Source and Binary jars.
>
> yup... I also believe that the maven2 plugin for eclipse is making
> people aware of why this is needed.
>
> Would also be nice if we could automate the extraction of such javadoc
> jars and provide a javadocs.apache.org thingy
>
> > * Stable. Files should not be easy to change.
>
> yup
>
> > Multiple repositories are needed:
> >
> > * ASF Release Repository.
> > * ASF Snapshot Repository. Deployed to from CI.
> > * ASF Development Repository. May contain 3rd party jars and manual
> > snapshots.
>
> Can you elaborate more on this?
I thought the repo was only for Apache stuff, so the ASF Development
Repository doesn't make sense
>
> > Some random ideas:
> >
> > * Monitoring. Have a script that mails this list when something is
> > added to the release or development repositories. Much like a cvs
> > commit in another project.
>
> great idea
>
> > * Tighten things up in a staggered way. Every couple of months we can
> > enforce something new. ie) All non-PGP signed jars will be removed by
> > X.
>
> +1, incrementality is good
>
> > * Pull the Maven repositories out of the mirroring system. It's
> > unnecessary. With the rsync being down atm, there's not a lot of
> > reason to avoid it. In fact, rm the ones in the maven repos and move
> > the ones in archive.apache to repo.apache.org. Need various
> > repo.apache.org directory names.
>
> I've had a wild idea the other day, inspired by some talks with some
> other digital library fellows that were complaining about bandwidth
> consumption. They pointed me at
>
> http://osprey.ibiblio.org/
>
> which is something that ibiblio does to allow transparent
> bittorrent-ization of resources.
>
> And I thought... hmmm, we could take Azureus (which is a kickass
> bittorrent implementation in java), strip down the transport part and
> integrate it in maven. Then use osprey on repo.apache.org to
> bittorrent-ize things.
Sounds cool, are you volunteering ;) ?
>
> Thoughts?
>
> --
> Stefano.
>
>
--
I could give you my word as a Spaniard.
No good. I've known too many Spaniards.
-- The Princess Bride
Re: [Plan of action] Setting up an official maven repository for
the ASF
Posted by Stefano Mazzocchi <st...@apache.org>.
Henri Yandell wrote:
> Nothing from Brett yet, so braindump from me.
Henry, thanks for your reply.
> Two classes of repository - m1 and m2. We should focus on short-term
> goals for the m1 one and long-term for the m2 one.
Agreed.
> Some goals:
>
> * Development ease. It should not take forever to get a dependency
> deployed.
yes, although if shouldn't make it so easy that people will want to do
it without reading the "readme".
> * Authentic. PGP .asc files for each release.
I've been thinking about this and I suspect that this requires some
Maven changes. I mean, what's the point of having a PGP file if maven
doesn't use it? and currently the POM doesn't have the metadata to
locate the official PGP file (which, remember, should always fetched
from the original location never from a mirror.
> * Official. Projects should know their code is being placed in the
> repository.
Yes and I think that the PGP signature should contain trace of who did
it, so that there is some trace.
> * Oversight. People shouldn't be deploying jars randomly.
if we do the above, we solve this as well: knowing that you can be
traced, random stuff would slow down.
> * Complete. Javadoc, Source and Binary jars.
yup... I also believe that the maven2 plugin for eclipse is making
people aware of why this is needed.
Would also be nice if we could automate the extraction of such javadoc
jars and provide a javadocs.apache.org thingy
> * Stable. Files should not be easy to change.
yup
> Multiple repositories are needed:
>
> * ASF Release Repository.
> * ASF Snapshot Repository. Deployed to from CI.
> * ASF Development Repository. May contain 3rd party jars and manual
> snapshots.
Can you elaborate more on this?
> Some random ideas:
>
> * Monitoring. Have a script that mails this list when something is
> added to the release or development repositories. Much like a cvs
> commit in another project.
great idea
> * Tighten things up in a staggered way. Every couple of months we can
> enforce something new. ie) All non-PGP signed jars will be removed by
> X.
+1, incrementality is good
> * Pull the Maven repositories out of the mirroring system. It's
> unnecessary. With the rsync being down atm, there's not a lot of
> reason to avoid it. In fact, rm the ones in the maven repos and move
> the ones in archive.apache to repo.apache.org. Need various
> repo.apache.org directory names.
I've had a wild idea the other day, inspired by some talks with some
other digital library fellows that were complaining about bandwidth
consumption. They pointed me at
http://osprey.ibiblio.org/
which is something that ibiblio does to allow transparent
bittorrent-ization of resources.
And I thought... hmmm, we could take Azureus (which is a kickass
bittorrent implementation in java), strip down the transport part and
integrate it in maven. Then use osprey on repo.apache.org to
bittorrent-ize things.
Thoughts?
--
Stefano.
Re: [Plan of action] Setting up an official maven repository for the ASF
Posted by Henri Yandell <fl...@gmail.com>.
Nothing from Brett yet, so braindump from me.
Two classes of repository - m1 and m2. We should focus on short-term
goals for the m1 one and long-term for the m2 one.
Some goals:
* Development ease. It should not take forever to get a dependency deployed.
* Authentic. PGP .asc files for each release.
* Official. Projects should know their code is being placed in the repository.
* Oversight. People shouldn't be deploying jars randomly.
* Complete. Javadoc, Source and Binary jars.
* Stable. Files should not be easy to change.
Multiple repositories are needed:
* ASF Release Repository.
* ASF Snapshot Repository. Deployed to from CI.
* ASF Development Repository. May contain 3rd party jars and manual snapshots.
Some random ideas:
* Monitoring. Have a script that mails this list when something is
added to the release or development repositories. Much like a cvs
commit in another project.
* Tighten things up in a staggered way. Every couple of months we can
enforce something new. ie) All non-PGP signed jars will be removed by
X.
* Pull the Maven repositories out of the mirroring system. It's
unnecessary. With the rsync being down atm, there's not a lot of
reason to avoid it. In fact, rm the ones in the maven repos and move
the ones in archive.apache to repo.apache.org. Need various
repo.apache.org directory names.
Hen
On 5/29/06, Stefano Mazzocchi <st...@apache.org> wrote:
> So, following up on my board@ message, here we should start to work on a
> plan of action for setting up an official maven repository for the ASF
> that would meet both board's legal concerns and infra's
> technical/security concerns and general social feasibility concerns.
>
> I know Brett has been thinking about this a lot so I'll leave the
> microphone to him to start.
>
> Let's rock.
>
> --
> Stefano.
>
>