You are viewing a plain text version of this content. The canonical link for it is here.
Posted to ivy-user@ant.apache.org by Archie Cobbs <ar...@dellroad.org> on 2008/04/15 17:06:13 UTC
Ivy RoundUp Repository - feedback requested
Hello fellow Ivy users,
I'd like to announce a new little project I've started, and ask for your
feedback (and help, if interested).
This project has two basic parts...
1. *Builder Resolver*<http://ivyroundup.googlecode.com/svn/wiki/files/builder.html>:
a new Ivy resolver that accesses ivy files and "build instructions" from an
online "builder" repository. "Builder" repositories contain ivy.xml files
but no artifacts. To get the artifacts, the build instructions are
downloaded from the repository and executed locally. These instructions
specify additional resource(s) to download and how to build the artifacts
from them, for example, by downloading a project's original distribution
archive directly from their web site and extracting the desired artifacts.
2. *Ivy RoundUp Repository* <http://ivyroundup.googlecode.com/>: an
online, open-source community "Builder" repository for all Ivy users.
Please click the links for more info and documentation.
I am lobbying to get the builder resolver added into Ivy itself; right now
it's still in patch form (you can download a pre-built ivy.jar from the
project website).
Some motivations for starting this project:
1. Ivyrep is no longer maintained, but we need a decent community Ivy
repository that everyone can share
2. Hosting hundreds of large files that are just copies of the same
files available elsewhere is expensive and redundant, so let's avoid doing
that
3. 99% of projects out there do not publish ivy.xml files, so we need
a community project that focuses on developing and maintaining them
4. To get the most out of Ivy, there needs to be a consistent set of
guidelines for creating ivy.xml files: how to choose organization names,
philosophy for defining configurations, etc. A community project supported
by Ivy users can provide this.
What I want to do is gauge interest in this idea and ask for any volunteers
who'd like to start adding and maintaining meta-data for their favorite
projects. The Ivy RoundUp repository is online now, though only as a
proof-of-concept (it only contains a few modules so far). Take a look and
you should be able to get the general idea:
http://ivyroundup.googlecode.com/
In the worst case, if nobody else is interested, I will just use this for
myself -- it's already working better than what I was doing (i.e., checking
in giant ZIP files into Subversion and creating a project for every one to
publish into our private Ivy repository), and in any case the work of
setting it up is already done. Note also anyone could create their own
private builder repository using this project as well.
In the best case, we'll put together a piece of infrastructure that all Ivy
users can really benefit from.
Let me know what you think.
Thanks,
-Archie
--
Archie L. Cobbs
Re: Ivy RoundUp Repository - feedback requested
Posted by Nicolas Lalevée <ni...@anyware-tech.com>.
Le jeudi 17 avril 2008, Gilles Scokart a écrit :
> Jing was not the only one to misunderstand the "building" term... I did
> also.
>
> Actually now the ivy roundup resolver as a resolver that allow you to
> take the meta-data from the roundup repository and take the artefact
> directly from the project distribution site!
>
> And then, I'm suddently much more enthousiast about it!
>
> It make me think to a thread that I read some time ago on the jmock
> mailing list [1].
> Initially, the jmock team didn't wanted to distribute themself their
> library in a maven repository. Their resistance was understandable
> and had a right justification. They would preffer someone else to
> redistribute. They finaly did it themself (after some external
> contributions) because of the pression of the maven users and because
> it is better for the user to receive the library directly from the
> team that build it.
>
> With the ivy roundup aproach the user has to trust the ivy roundup
> community to setup the right meta-data (as a user, I would have a to
> look at how this community work, but It is possible that I could trust
> them), and then I would have to trust the author of the library (and
> if I use their library, I guess that I trust them.
>
> With a maven2 redistribution aproach, I need to to trust the maven2
> community for both the meta-data and the binaries. It looks to me
> more "risky" to trust.
>
>
> All that to say : you should remove the image of "building". The
> resolver is a tool to get dependencies from their original
> distributions site. That should be reflected in the term you use if
> you want to keep me enthousiastic :-)
>
> [1]
> http://article.gmane.org/gmane.comp.java.jmock.user/972/match=maven+downstr
>eam+release
This idea behind this repository is getting more clear to me too. :)
Then I cannot not compare it to Linux distributions. It reminds me an article
about debian packager, about the different responsabilities about developping
and packaging.
Then I wouldn't make a difference in trust between the maven community and an
ivy round up community. Their main job is about packaging, so we have to
trust both for that. If I can trust the ivy roundup community to produce a
reproductible build, then I would trust the maven community to execute a
reproductible build.
This make me think of gentoo VS ubuntu. The first one is just packaging build
script, the other one is packaging build. It sometimes can make sense in the
native world, but not in a jvm world.
On the other hand, maintaing build scripts is definitively a good idea. Just
like the debian/ubuntu community is maintaing their build scripts. As said
Xavier it helps the community to maintain a module along with the upgrading
versions. I would consider the Ivy RoundUp Repository being a repository for
maintainers more than for Ivy users. I think that Ivy users should only see
jars.
That discussion makes me also think of the Orbit community. There are building
and releasing entire osgi bundle repositories. See for instance:
http://download.eclipse.org/tools/orbit/committers/drops/I20080416020842
Thought I don't know how practical it is.
Nicolas
>
> On 17/04/2008, Xavier Hanin <xa...@gmail.com> wrote:
> > On Wed, Apr 16, 2008 at 8:54 PM, Archie Cobbs <ar...@dellroad.org> wrote:
> > > On Wed, Apr 16, 2008 at 1:17 PM, Jing Xue <ji...@digizenstudio.com>
> > >
> > > wrote:
> > > > I read the thread on the dev list, and was thoroughly interested in
> > > > the idea of a community-maintained public ivy.xml repository,
> > > > because a carefully defined ivy.xml provides a lot more value than
> > > > the ones
> > >
> > > generated
> > >
> > > > from the pom, and would be something worth sharing.
> > > >
> > > > On the other hand, I'm not sure I see the value in the builder
> > > > resolver.
> > >
> > > I
> > >
> > > > guess I'm having a hard time imagining a use case where I would
> > > > prefer building a 3rd party artifact locally rather than simply
> > > > downloading it through an ibiblio resolver, or from a company-level
> > > > proxy repository.
> > >
> > > The problem is that not all artifacts are available as URLs. Many
> > > times, they are packaged up inside a tar.gz file and the only way to
> > > get at them is
> > > to extract them.
> >
> > I think the problem is in the intention and the meaning of "building"
> > artifacts. If I understand correctly, Jing think that the builder
> > resolver will actually compile and package the artifacts, which wouldn't
> > make a lot of sense. What the "builder" actually do is simply download
> > the artifacts from where they are over the net, and when the jar is
> > actually contained in an archive, do the extract. For some artifacts like
> > javadoc and sources which are not always provided as an artifact usable
> > by IDE, the builder can actually extract the zip where the source files
> > can be obtained, and repackage the source files in a zip which can be
> > used by an IDE. For instance if you want to create a module for Ivy
> > 1.4.1, you could get the zips from
> > <http://www.jaya.free.fr/downloads/ivy/1.4.1/>, then unpack them to get
> > the main Ivy jars, and repack the sources and javadocs as usable
> > artifacts for IDEs.
> >
> > The big advantage to do this automatically instead of manually is that
> > the work needed when a new minor version is released is usually minor:
> > bump up the version and update the sha1 and if the packaging hasn't
> > changed you're done! Then you can concentrate on metadata.
> >
> >
> > Xavier
> >
> > > So without builder resolver, someone has to:
> > >
> > > 1. Build and maintain an online repository that has huge disk space
> > > (and bandwidth) requirements
> > > 2. Perform the manual download, extract, and publish steps every
> > > time a software has a new release
> > >
> > > With builder resolver, #1 goes away entirely and #2 is reduced to
> > > updating a
> > > SHA1 checksum (common case).
> > >
> > > If you don't think #1 and #2 matter, then why aren't you volunteering
> > > to do
> > > them for us? :-)
> > >
> > > -Archie
> > >
> > > --
> > > Archie L. Cobbs
> >
> > --
> > Xavier Hanin - Independent Java Consultant
> > http://xhab.blogspot.com/
> > http://ant.apache.org/ivy/
> > http://www.xoocode.org/
--
Nicolas LALEVÉE
ANYWARE TECHNOLOGIES
Tel : +33 (0)5 61 00 52 90
Fax : +33 (0)5 61 00 51 46
http://www.anyware-tech.com
Re: Ivy RoundUp Repository - feedback requested
Posted by Gilles Scokart <gs...@gmail.com>.
Jing was not the only one to misunderstand the "building" term... I did also.
Actually now the ivy roundup resolver as a resolver that allow you to
take the meta-data from the roundup repository and take the artefact
directly from the project distribution site!
And then, I'm suddently much more enthousiast about it!
It make me think to a thread that I read some time ago on the jmock
mailing list [1].
Initially, the jmock team didn't wanted to distribute themself their
library in a maven repository. Their resistance was understandable
and had a right justification. They would preffer someone else to
redistribute. They finaly did it themself (after some external
contributions) because of the pression of the maven users and because
it is better for the user to receive the library directly from the
team that build it.
With the ivy roundup aproach the user has to trust the ivy roundup
community to setup the right meta-data (as a user, I would have a to
look at how this community work, but It is possible that I could trust
them), and then I would have to trust the author of the library (and
if I use their library, I guess that I trust them.
With a maven2 redistribution aproach, I need to to trust the maven2
community for both the meta-data and the binaries. It looks to me
more "risky" to trust.
All that to say : you should remove the image of "building". The
resolver is a tool to get dependencies from their original
distributions site. That should be reflected in the term you use if
you want to keep me enthousiastic :-)
[1] http://article.gmane.org/gmane.comp.java.jmock.user/972/match=maven+downstream+release
On 17/04/2008, Xavier Hanin <xa...@gmail.com> wrote:
> On Wed, Apr 16, 2008 at 8:54 PM, Archie Cobbs <ar...@dellroad.org> wrote:
>
> > On Wed, Apr 16, 2008 at 1:17 PM, Jing Xue <ji...@digizenstudio.com>
> > wrote:
> >
> > > I read the thread on the dev list, and was thoroughly interested in the
> > > idea of a community-maintained public ivy.xml repository, because a
> > > carefully defined ivy.xml provides a lot more value than the ones
> > generated
> > > from the pom, and would be something worth sharing.
> > >
> > > On the other hand, I'm not sure I see the value in the builder resolver.
> > I
> > > guess I'm having a hard time imagining a use case where I would prefer
> > > building a 3rd party artifact locally rather than simply downloading it
> > > through an ibiblio resolver, or from a company-level proxy repository.
> >
> >
> > The problem is that not all artifacts are available as URLs. Many times,
> > they are packaged up inside a tar.gz file and the only way to get at them
> > is
> > to extract them.
>
>
> I think the problem is in the intention and the meaning of "building"
> artifacts. If I understand correctly, Jing think that the builder resolver
> will actually compile and package the artifacts, which wouldn't make a lot
> of sense. What the "builder" actually do is simply download the artifacts
> from where they are over the net, and when the jar is actually contained in
> an archive, do the extract. For some artifacts like javadoc and sources
> which are not always provided as an artifact usable by IDE, the builder can
> actually extract the zip where the source files can be obtained, and
> repackage the source files in a zip which can be used by an IDE. For
> instance if you want to create a module for Ivy 1.4.1, you could get the
> zips from <http://www.jaya.free.fr/downloads/ivy/1.4.1/>, then unpack them
> to get the main Ivy jars, and repack the sources and javadocs as usable
> artifacts for IDEs.
>
> The big advantage to do this automatically instead of manually is that the
> work needed when a new minor version is released is usually minor: bump up
> the version and update the sha1 and if the packaging hasn't changed you're
> done! Then you can concentrate on metadata.
>
>
> Xavier
>
>
>
> >
> > So without builder resolver, someone has to:
> >
> > 1. Build and maintain an online repository that has huge disk space
> > (and bandwidth) requirements
> > 2. Perform the manual download, extract, and publish steps every time
> > a software has a new release
> >
> > With builder resolver, #1 goes away entirely and #2 is reduced to updating
> > a
> > SHA1 checksum (common case).
> >
> > If you don't think #1 and #2 matter, then why aren't you volunteering to
> > do
> > them for us? :-)
> >
> > -Archie
> >
> > --
> > Archie L. Cobbs
> >
>
>
>
>
> --
> Xavier Hanin - Independent Java Consultant
> http://xhab.blogspot.com/
> http://ant.apache.org/ivy/
> http://www.xoocode.org/
>
--
Gilles Scokart
Re: Ivy RoundUp Repository - feedback requested
Posted by Xavier Hanin <xa...@gmail.com>.
On Wed, Apr 16, 2008 at 8:54 PM, Archie Cobbs <ar...@dellroad.org> wrote:
> On Wed, Apr 16, 2008 at 1:17 PM, Jing Xue <ji...@digizenstudio.com>
> wrote:
>
> > I read the thread on the dev list, and was thoroughly interested in the
> > idea of a community-maintained public ivy.xml repository, because a
> > carefully defined ivy.xml provides a lot more value than the ones
> generated
> > from the pom, and would be something worth sharing.
> >
> > On the other hand, I'm not sure I see the value in the builder resolver.
> I
> > guess I'm having a hard time imagining a use case where I would prefer
> > building a 3rd party artifact locally rather than simply downloading it
> > through an ibiblio resolver, or from a company-level proxy repository.
>
>
> The problem is that not all artifacts are available as URLs. Many times,
> they are packaged up inside a tar.gz file and the only way to get at them
> is
> to extract them.
I think the problem is in the intention and the meaning of "building"
artifacts. If I understand correctly, Jing think that the builder resolver
will actually compile and package the artifacts, which wouldn't make a lot
of sense. What the "builder" actually do is simply download the artifacts
from where they are over the net, and when the jar is actually contained in
an archive, do the extract. For some artifacts like javadoc and sources
which are not always provided as an artifact usable by IDE, the builder can
actually extract the zip where the source files can be obtained, and
repackage the source files in a zip which can be used by an IDE. For
instance if you want to create a module for Ivy 1.4.1, you could get the
zips from <http://www.jaya.free.fr/downloads/ivy/1.4.1/>, then unpack them
to get the main Ivy jars, and repack the sources and javadocs as usable
artifacts for IDEs.
The big advantage to do this automatically instead of manually is that the
work needed when a new minor version is released is usually minor: bump up
the version and update the sha1 and if the packaging hasn't changed you're
done! Then you can concentrate on metadata.
Xavier
>
> So without builder resolver, someone has to:
>
> 1. Build and maintain an online repository that has huge disk space
> (and bandwidth) requirements
> 2. Perform the manual download, extract, and publish steps every time
> a software has a new release
>
> With builder resolver, #1 goes away entirely and #2 is reduced to updating
> a
> SHA1 checksum (common case).
>
> If you don't think #1 and #2 matter, then why aren't you volunteering to
> do
> them for us? :-)
>
> -Archie
>
> --
> Archie L. Cobbs
>
--
Xavier Hanin - Independent Java Consultant
http://xhab.blogspot.com/
http://ant.apache.org/ivy/
http://www.xoocode.org/
Re: Ivy RoundUp Repository - feedback requested
Posted by Archie Cobbs <ar...@dellroad.org>.
On Sat, Apr 19, 2008 at 2:03 AM, Xavier Hanin <xa...@gmail.com>
wrote:
> > So far, my understanding is that those issues are being addressed by the
> > ivy community with ivy files manually created, optimized and put in some
> > proxy repository (or a local file system resolver). A proxy repository
> > can also be used to provide artifacts that aren't directly available
> > from the public repositories. The proxy repo maintainer can (and perhaps
> > should) take up the task of preparing these artifacts. The good thing
> > about this approach is, they only need to be prepared once and then kept
> > in the proxy for good, instead of having to be involved in every build
> > process.
>
> Good, but to maintain your proxy when a new version is out you have to
> reproduce very similar steps. I think the idea of roundup repo comes from
> this experience, feeling that you lose time because you're missing
> "distilling" instructions. So with Ivy roundup it's super easy to setup a
> proxy: the proxy use Ivy roundup, the end user use the proxy. It's not the
> approach taken yet, because most people starting to use a dependency
> manager
> don't want to have to setup a proxy or server or whatever. They want
> something that works out of the box with resources available over the net.
> That's why I think the roudup repo AND the "builder" resolver make sense.
> Now I agree that pretty quickly we'll need some other tools than the
> "builder" resolver to access the repo. But I think the value is not in the
> resolver, the value is in the repository of Ivy files AND "distilling"
> instructions.
>
> > So, the proxy-based approach works well enough at least for me. And that
> > is why I said earlier that I would be immensely interested in a
> > community-maintained ivy metadata repository that does not require a
> > special resolver to access, i.e., can be hooked in with a dual resolver.
> > That would make it a great _and_ easily accessible channel for all
> > the ivy users to share manually optimized metadata.
>
> With only Ivy files, you can hit the problem of naming inconsistencies.
> Ivy
> namespaces can help, but I'm not sure it can address all problems. The
> roundup approach has no limitation, except the time required to create the
> metadata.
>
>
To give an update to the list...
We've been doing a lot of work on the Ivy RoundUp repository and things are
moving along nicely. One important new feature we've added is support for
building a normal repository from the RoundUp meta-data.
We're starting to look for people to start really using it as well, and
(hopefully) contribute suggestions and new modules/revisions that you may
have, or want, etc.
There is a also a mailing list set up now. Join us there if you're
interested.
Details on all of this are on the project site:
http://ivyroundup.googlecode.com/
Thanks,
-Archie
--
Archie L. Cobbs
Re: Ivy RoundUp Repository - feedback requested
Posted by Xavier Hanin <xa...@gmail.com>.
On Sat, Apr 19, 2008 at 6:55 AM, Jing Xue <ji...@digizenstudio.com> wrote:
> On Wed, Apr 16, 2008 at 01:54:42PM -0500, Archie Cobbs wrote:
> >
> > The problem is that not all artifacts are available as URLs. Many times,
> > they are packaged up inside a tar.gz file and the only way to get at
> them is
> > to extract them.
> >
> > So without builder resolver, someone has to:
> >
> > 1. Build and maintain an online repository that has huge disk space
> > (and bandwidth) requirements
> > 2. Perform the manual download, extract, and publish steps every time
> > a software has a new release
> >
> > With builder resolver, #1 goes away entirely and #2 is reduced to
> updating a
> > SHA1 checksum (common case).
> >
> > If you don't think #1 and #2 matter, then why aren't you volunteering to
> do
> > them for us? :-)
>
> Well I just sent you an email to apply for joining the project, because
> I felt that I could help out in maintaining the repo. 8-)
>
> I must confess, however, even after all the clarification from everyone
> about what "build" means, I'm still not 100% convinced about the builder
> resolver. Let me try and explain:
>
> There is already the maven2 repository that everybody using ivy these
> days pretty much relies on. So most - 99% - of the artifacts are already
I don't think it's that close to 100%. Some modules are missing due to
license restriction, others are missing because their developer now have
their own repo, which is not synced with maven central. So I'd rather say
that 99% are available in "one" maven2 repo somewhere. And this makes a
difference, especially for newbies. With Ivy roundup we can centralize
modules coming from several sources, most of which will be maven 2 repo,
others will be download sites. Sometime it will be mixed, when only the
binary jar is available on a maven 2 repo, and the source and javadocs can
only be obtained through a download site.
>
> there, instead of having to be downloaded and extracted from the
> original distribution packages. Speaking from my experience, I have only
> had a few occasions where I had to manually download jars and put them
> in my proxy repository - for instance, the notorious jta jar that
> ibiblio cannot legally redistribute, thanks to Sun's license terms.
jta.jar is not available java.net maven repository, so we can even provide
access to this one in IvyRoundUp.
>
>
> On the other hand, IMHO, the gap between an ideal ivy user experience
> and the current state of affair comes from two issues:
>
> 1. The inconsistent quality of the metadata files in the maven2
> repositories. Sometimes they are simply broken, sometimes else they are
> not carefully written and come with, for example, unnecessary
> dependencies.
Agreed. And don't forget name inconsistencies, which is a real pain for
newcomers.
>
>
> 2. The limitations posed by the POM syntax itself leads to
> overly-simplified pom.xml's, which in turn prevents the auto-generated
> ivy files from leveraging all the expressiveness that ivy provides.
+1
>
>
> So far, my understanding is that those issues are being addressed by the
> ivy community with ivy files manually created, optimized and put in some
> proxy repository (or a local file system resolver). A proxy repository
> can also be used to provide artifacts that aren't directly available
> from the public repositories. The proxy repo maintainer can (and perhaps
> should) take up the task of preparing these artifacts. The good thing
> about this approach is, they only need to be prepared once and then kept
> in the proxy for good, instead of having to be involved in every build
> process.
Good, but to maintain your proxy when a new version is out you have to
reproduce very similar steps. I think the idea of roundup repo comes from
this experience, feeling that you lose time because you're missing
"distilling" instructions. So with Ivy roundup it's super easy to setup a
proxy: the proxy use Ivy roundup, the end user use the proxy. It's not the
approach taken yet, because most people starting to use a dependency manager
don't want to have to setup a proxy or server or whatever. They want
something that works out of the box with resources available over the net.
That's why I think the roudup repo AND the "builder" resolver make sense.
Now I agree that pretty quickly we'll need some other tools than the
"builder" resolver to access the repo. But I think the value is not in the
resolver, the value is in the repository of Ivy files AND "distilling"
instructions.
>
>
> So, the proxy-based approach works well enough at least for me. And that
> is why I said earlier that I would be immensely interested in a
> community-maintained ivy metadata repository that does not require a
> special resolver to access, i.e., can be hooked in with a dual resolver.
> That would make it a great _and_ easily accessible channel for all
> the ivy users to share manually optimized metadata.
With only Ivy files, you can hit the problem of naming inconsistencies. Ivy
namespaces can help, but I'm not sure it can address all problems. The
roundup approach has no limitation, except the time required to create the
metadata.
Xavier
>
>
> Cheers.
> --
> Jing Xue
>
>
--
Xavier Hanin - Independent Java Consultant
http://xhab.blogspot.com/
http://ant.apache.org/ivy/
http://www.xoocode.org/
Re: Ivy RoundUp Repository - feedback requested
Posted by Jing Xue <ji...@digizenstudio.com>.
On Wed, Apr 16, 2008 at 01:54:42PM -0500, Archie Cobbs wrote:
>
> The problem is that not all artifacts are available as URLs. Many times,
> they are packaged up inside a tar.gz file and the only way to get at them is
> to extract them.
>
> So without builder resolver, someone has to:
>
> 1. Build and maintain an online repository that has huge disk space
> (and bandwidth) requirements
> 2. Perform the manual download, extract, and publish steps every time
> a software has a new release
>
> With builder resolver, #1 goes away entirely and #2 is reduced to updating a
> SHA1 checksum (common case).
>
> If you don't think #1 and #2 matter, then why aren't you volunteering to do
> them for us? :-)
Well I just sent you an email to apply for joining the project, because
I felt that I could help out in maintaining the repo. 8-)
I must confess, however, even after all the clarification from everyone
about what "build" means, I'm still not 100% convinced about the builder
resolver. Let me try and explain:
There is already the maven2 repository that everybody using ivy these
days pretty much relies on. So most - 99% - of the artifacts are already
there, instead of having to be downloaded and extracted from the
original distribution packages. Speaking from my experience, I have only
had a few occasions where I had to manually download jars and put them
in my proxy repository - for instance, the notorious jta jar that
ibiblio cannot legally redistribute, thanks to Sun's license terms.
On the other hand, IMHO, the gap between an ideal ivy user experience
and the current state of affair comes from two issues:
1. The inconsistent quality of the metadata files in the maven2
repositories. Sometimes they are simply broken, sometimes else they are
not carefully written and come with, for example, unnecessary
dependencies.
2. The limitations posed by the POM syntax itself leads to
overly-simplified pom.xml's, which in turn prevents the auto-generated
ivy files from leveraging all the expressiveness that ivy provides.
So far, my understanding is that those issues are being addressed by the
ivy community with ivy files manually created, optimized and put in some
proxy repository (or a local file system resolver). A proxy repository
can also be used to provide artifacts that aren't directly available
from the public repositories. The proxy repo maintainer can (and perhaps
should) take up the task of preparing these artifacts. The good thing
about this approach is, they only need to be prepared once and then kept
in the proxy for good, instead of having to be involved in every build
process.
So, the proxy-based approach works well enough at least for me. And that
is why I said earlier that I would be immensely interested in a
community-maintained ivy metadata repository that does not require a
special resolver to access, i.e., can be hooked in with a dual resolver.
That would make it a great _and_ easily accessible channel for all
the ivy users to share manually optimized metadata.
Cheers.
--
Jing Xue
Re: Ivy RoundUp Repository - feedback requested
Posted by Archie Cobbs <ar...@dellroad.org>.
On Wed, Apr 16, 2008 at 1:17 PM, Jing Xue <ji...@digizenstudio.com> wrote:
> I read the thread on the dev list, and was thoroughly interested in the
> idea of a community-maintained public ivy.xml repository, because a
> carefully defined ivy.xml provides a lot more value than the ones generated
> from the pom, and would be something worth sharing.
>
> On the other hand, I'm not sure I see the value in the builder resolver. I
> guess I'm having a hard time imagining a use case where I would prefer
> building a 3rd party artifact locally rather than simply downloading it
> through an ibiblio resolver, or from a company-level proxy repository.
The problem is that not all artifacts are available as URLs. Many times,
they are packaged up inside a tar.gz file and the only way to get at them is
to extract them.
So without builder resolver, someone has to:
1. Build and maintain an online repository that has huge disk space
(and bandwidth) requirements
2. Perform the manual download, extract, and publish steps every time
a software has a new release
With builder resolver, #1 goes away entirely and #2 is reduced to updating a
SHA1 checksum (common case).
If you don't think #1 and #2 matter, then why aren't you volunteering to do
them for us? :-)
-Archie
--
Archie L. Cobbs
Re: Ivy RoundUp Repository - feedback requested
Posted by Jing Xue <ji...@digizenstudio.com>.
Quoting Archie Cobbs <ar...@dellroad.org>:
> Hello fellow Ivy users,
>
> I'd like to announce a new little project I've started, and ask for your
> feedback (and help, if interested).
>
> This project has two basic parts...
>
> 1. *Builder
> Resolver*<http://ivyroundup.googlecode.com/svn/wiki/files/builder.html>:
> a new Ivy resolver that accesses ivy files and "build
> instructions" from an
> online "builder" repository. "Builder" repositories contain ivy.xml files
> but no artifacts. To get the artifacts, the build instructions are
> downloaded from the repository and executed locally. These instructions
> specify additional resource(s) to download and how to build the artifacts
> from them, for example, by downloading a project's original distribution
> archive directly from their web site and extracting the desired artifacts.
> 2. *Ivy RoundUp Repository* <http://ivyroundup.googlecode.com/>: an
> online, open-source community "Builder" repository for all Ivy users.
[snip]
I read the thread on the dev list, and was thoroughly interested in
the idea of a community-maintained public ivy.xml repository, because
a carefully defined ivy.xml provides a lot more value than the ones
generated from the pom, and would be something worth sharing.
On the other hand, I'm not sure I see the value in the builder
resolver. I guess I'm having a hard time imagining a use case where I
would prefer building a 3rd party artifact locally rather than simply
downloading it through an ibiblio resolver, or from a company-level
proxy repository.
Cheers.
--
Jing Xue
Re: Ivy RoundUp Repository - feedback requested
Posted by Xavier Hanin <xa...@gmail.com>.
On Thu, Apr 17, 2008 at 8:59 PM, Archie Cobbs <ar...@dellroad.org> wrote:
> On Thu, Apr 17, 2008 at 1:10 PM, Xavier Hanin <xa...@gmail.com>
> wrote:
>
> > It depends how you consider the roundup repository. I think its intent
> is
> > to
> > be a kind of meta repository: it contains metadata helping to build an
> > actual repository. Maybe what's confusing is that archie suggest to use
> it
> > as a repository directly. IMO this is required only until we can get
> > enough
> > support to have a good hosting for the actual repository. Then both the
> > meta
> > repository and the actual repository would make sense: the meta
> repository
> > is helpful for people who want to create their own repository, and
> easier
> > to
> > update than a repository made by hand. But it isn't used directly from
> > Ivy.
> > The actual repository is the only thing that is used by mere users.
> >
>
> There are actually three possibilities...
>
> 1. Builder Repository: repository contains ivy.xml and builder.xml
> files only, and the user is required to use the builder resolver,
> whereby
> the actual artifacts are downloaded and extracted as necessary at
> resolution
> time.
> 2. Builder Repository + Artifact repository: same as above, but you
> configure your resolver with the "resourceURL" attribute (which
> overrides
> the normal resource URLs), which points to the Artifact repository
> containing all of the ZIP/TGZ files that you need to download. The
> Artifact
> repository could be local or community/online. This means we don't have
> to
> rely on each/every project's server being reliable.
> 3. Build-your-own-Repository: we would add a new ant task in the Ivy
> RoundUp build.xml that actually executes all the builder.xml
> download/extract instructions for every module in the repository to get
> all
> the artifacts and then combines those artifacts with the ivy.xml files
> to
> create a normal Ivy repository.
>
> Correlating with Xavier's comments: #3 == "meta repository" whereas #1 =
> "repository directly".
>
> The important concept here is *separation of the meta-data from the data*.
>
> Also important is *not forcing the requirement for a high capacity, high
> bandwidth public server*.... I'll repeat my claim that this is important
> based on the evidence that nobody has yet volunteered to replace Ivyrep.
>
> If someone is willing to host a normal repository, then options #2 or #3
> make sense. If not, then #1 makes sense. But it is flexible enough to
> work
> in any/all of these ways. I think we should start with #1 and then we can
> move to #2 or #3 if and when someone volunteers the additional servers and
> disk space.
+1. Starting with #1 is the best option right now, then we'll later move to
either #2 or #3 (I prefer #3, but it's only my opinion).
Xavier
>
>
> -Archie
>
> --
> Archie L. Cobbs
>
--
Xavier Hanin - Independent Java Consultant
http://xhab.blogspot.com/
http://ant.apache.org/ivy/
http://www.xoocode.org/
Re: Ivy RoundUp Repository - feedback requested
Posted by Archie Cobbs <ar...@dellroad.org>.
On Thu, Apr 17, 2008 at 1:10 PM, Xavier Hanin <xa...@gmail.com>
wrote:
> It depends how you consider the roundup repository. I think its intent is
> to
> be a kind of meta repository: it contains metadata helping to build an
> actual repository. Maybe what's confusing is that archie suggest to use it
> as a repository directly. IMO this is required only until we can get
> enough
> support to have a good hosting for the actual repository. Then both the
> meta
> repository and the actual repository would make sense: the meta repository
> is helpful for people who want to create their own repository, and easier
> to
> update than a repository made by hand. But it isn't used directly from
> Ivy.
> The actual repository is the only thing that is used by mere users.
>
There are actually three possibilities...
1. Builder Repository: repository contains ivy.xml and builder.xml
files only, and the user is required to use the builder resolver, whereby
the actual artifacts are downloaded and extracted as necessary at resolution
time.
2. Builder Repository + Artifact repository: same as above, but you
configure your resolver with the "resourceURL" attribute (which overrides
the normal resource URLs), which points to the Artifact repository
containing all of the ZIP/TGZ files that you need to download. The Artifact
repository could be local or community/online. This means we don't have to
rely on each/every project's server being reliable.
3. Build-your-own-Repository: we would add a new ant task in the Ivy
RoundUp build.xml that actually executes all the builder.xml
download/extract instructions for every module in the repository to get all
the artifacts and then combines those artifacts with the ivy.xml files to
create a normal Ivy repository.
Correlating with Xavier's comments: #3 == "meta repository" whereas #1 =
"repository directly".
The important concept here is *separation of the meta-data from the data*.
Also important is *not forcing the requirement for a high capacity, high
bandwidth public server*.... I'll repeat my claim that this is important
based on the evidence that nobody has yet volunteered to replace Ivyrep.
If someone is willing to host a normal repository, then options #2 or #3
make sense. If not, then #1 makes sense. But it is flexible enough to work
in any/all of these ways. I think we should start with #1 and then we can
move to #2 or #3 if and when someone volunteers the additional servers and
disk space.
-Archie
--
Archie L. Cobbs
Re: Ivy RoundUp Repository - feedback requested
Posted by Xavier Hanin <xa...@gmail.com>.
On Thu, Apr 17, 2008 at 6:06 PM, Brown, Carlton <
Carlton.Brown@compucredit.com> wrote:
> > -----Original Message-----
> > From: archie.cobbs@gmail.com [mailto:archie.cobbs@gmail.com]
> > On Behalf Of Archie Cobbs
> > Sent: Thursday, April 17, 2008 10:31 AM
> > To: ivy-user@ant.apache.org
> > Subject: Re: Ivy RoundUp Repository - feedback requested
> >
> > When I started implementing this resolver, I tried to think
> > about "building"
> > as broadly as possible. Then later, for security reasons (and
> > because get/unzip/etc. is all that's really needed), it
> > seemed better to limit what the "build" process can do to
> > simply extracting files and/or repackaging them. So really
> > now it perhaps should be called the "extractor" resolver or something.
>
> IMO, I would go as far as tackling the extraction problem, but not
> farther than that. Ivy is very configurable from Ant, which can handle
> all your build needs.
>
> I could see maybe adding an aggregate attribute to the existing
> resolvers, like so.
> <url name="foo-zips">
> <archive type="tar" compression="gz"
> pattern="${url}/[organisation]/[module].tar.gz"/>
> <artifact pattern="[organisation]/[module]/[artifact].[ext]"/>
> <ivy pattern="[organisation]/[module]/ivy.xml"/>
> </url>
>
> Thus the caller would never need to know that the artifacts came from a
> ..tar.gz. I think that is a good idea. However, I'm not sure it's a
> good idea to introduce other concerns around building. One reason I
> prefer Ivy+Ant to Maven is that I don't have to clutter my dependency
> descriptor with a bunch of goop for source-mapping and build logic, and
> I would hope that any public repository would preserve and respect that
> principle.
It depends how you consider the roundup repository. I think its intent is to
be a kind of meta repository: it contains metadata helping to build an
actual repository. Maybe what's confusing is that archie suggest to use it
as a repository directly. IMO this is required only until we can get enough
support to have a good hosting for the actual repository. Then both the meta
repository and the actual repository would make sense: the meta repository
is helpful for people who want to create their own repository, and easier to
update than a repository made by hand. But it isn't used directly from Ivy.
The actual repository is the only thing that is used by mere users.
In that case I'm not sure it wouldn't make sense to allow actual compilation
of modules from the metadata provided by the meta repository. Not sure it
should be part of the resolver though, and this can come later IMO.
Xavier
>
> -----------------------------------------
> ====================================================
> This message contains PRIVILEGED and CONFIDENTIAL
> information that is intended only for use by the
> named recipient. If you are not the named recipient,
> any disclosure, dissemination, or action based on
> the contents of this message is prohibited. In such
> case please notify us and destroy and delete all
> copies of this transmission. Thank you.
> ====================================================
>
--
Xavier Hanin - Independent Java Consultant
http://xhab.blogspot.com/
http://ant.apache.org/ivy/
http://www.xoocode.org/
RE: Ivy RoundUp Repository - feedback requested
Posted by "Brown, Carlton" <Ca...@compucredit.com>.
> -----Original Message-----
> From: archie.cobbs@gmail.com [mailto:archie.cobbs@gmail.com]
> On Behalf Of Archie Cobbs
> Sent: Thursday, April 17, 2008 10:31 AM
> To: ivy-user@ant.apache.org
> Subject: Re: Ivy RoundUp Repository - feedback requested
>
> When I started implementing this resolver, I tried to think
> about "building"
> as broadly as possible. Then later, for security reasons (and
> because get/unzip/etc. is all that's really needed), it
> seemed better to limit what the "build" process can do to
> simply extracting files and/or repackaging them. So really
> now it perhaps should be called the "extractor" resolver or something.
IMO, I would go as far as tackling the extraction problem, but not
farther than that. Ivy is very configurable from Ant, which can handle
all your build needs.
I could see maybe adding an aggregate attribute to the existing
resolvers, like so.
<url name="foo-zips">
<archive type="tar" compression="gz"
pattern="${url}/[organisation]/[module].tar.gz"/>
<artifact pattern="[organisation]/[module]/[artifact].[ext]"/>
<ivy pattern="[organisation]/[module]/ivy.xml"/>
</url>
Thus the caller would never need to know that the artifacts came from a
..tar.gz. I think that is a good idea. However, I'm not sure it's a
good idea to introduce other concerns around building. One reason I
prefer Ivy+Ant to Maven is that I don't have to clutter my dependency
descriptor with a bunch of goop for source-mapping and build logic, and
I would hope that any public repository would preserve and respect that
principle.
-----------------------------------------
====================================================
This message contains PRIVILEGED and CONFIDENTIAL
information that is intended only for use by the
named recipient. If you are not the named recipient,
any disclosure, dissemination, or action based on
the contents of this message is prohibited. In such
case please notify us and destroy and delete all
copies of this transmission. Thank you.
====================================================
Re: Ivy RoundUp Repository - feedback requested
Posted by Archie Cobbs <ar...@dellroad.org>.
Thanks for your feedback!
On Thu, Apr 17, 2008 at 4:40 AM, Gilles Scokart <gs...@gmail.com> wrote:
> 1. I like the fact that the ivy.xml and build.xml contains a license
> boilerplate. I think it should a rule, and that the AL should be
> applied systematically for all meta-data info stored in ivy-rep
>
Yes, this is easy to achieve during the repository build process. We also do
XSD checks, etc.
> 2. As I said before, I don't like the term 'build'. Something like
> 'download', 'install', 'get', etc... would be much better.
>
My apologies about the naming... I didn't realize that there was so much
confusion.
When I started implementing this resolver, I tried to think about "building"
as broadly as possible. Then later, for security reasons (and because
get/unzip/etc. is all that's really needed), it seemed better to limit what
the "build" process can do to simply extracting files and/or repackaging
them. So really now it perhaps should be called the "extractor" resolver or
something.
In any case, because it uses ant to "build" the artifacts, in theory any
more general "build" process is possible. For example, occasionally I've
seen someone publish a small, one-class Java utility simply putting the
source file up on the web somewhere. To use this, it has to be compiled and
packaged into a jar manually. A more general "builder" resolver could do
this for you.
Those oddball cases are rare, so I'm happy for now restricting the "builder"
to just being an "extractor" or whatever. But in general, there may be cases
where more a general "building" capability is needed.
There could also be configuration knobs to allow this, e.g.,
allowGeneralBuild="true" or somesuch. Also digital signatures would help
make this safer.
> 3. Signatures should be considered, and all meta-data files should be
> signed.
>
Yes this idea was mentioned before and certainly makes sense.
> 4. You should consider versioning of the meta-data. ivyroundup will
> be a downstream distributor, you can make errors that you will have to
> fix, without asking a new release from your upstream distributor, and
> still allowing build reproduceability.
> (by build reproduceability, I mean that the user must be able to
> define with high precision what he want to change).
>
This will happen naturally, and follow the trunk, branches, and tags in SVN.
E.g. we may release and tag "Ivy RoundUp 1.0.3", which will persist
indefintely at an unique URL. Any user who points to that URL will always
get the same thing. When the user wants to "upgrade", they change their URL
to point to a newer release (or branch, if they want continuous updates).
> 5. A dependency repository needs to :
> - Notify users in case of a new version of a module
> that he use.
> - Provides release notes links.
>
Yes, this could follow the conventions of any project: mailing list, "News"
page, etc. Each release would have release notes. All very standard...
> 6. You might encounter hosting issues
>
Not sure what you mean here...
Thanks,
-Archie
--
Archie L. Cobbs
Re: Ivy RoundUp Repository - feedback requested
Posted by Niklas Matthies <ml...@nmhq.net>.
On Thu 2008-04-17 at 07:25h, Matt Benson wrote on ivy-user:
> --- Gilles Scokart <gs...@gmail.com> wrote:
:
> > 2. As I said before, I don't like the term 'build'. Something like
> > 'download', 'install', 'get', etc... would be much better.
>
> To chime in here, I'd like to point out that in some
> extreme hypothetical case a project might release in
> source form only. In this case a build _would_ take
> place. I'd like to see the RoundUp concept be
> reusable within an organization, where the build/get
> instructions might be to pull a particular svn tag,
> then execute an Ant build. So maybe "build" assumes
> too much, but we could explore some middleground that
> simply implies that some processing must take place...
> "prepare"?
How about "distill"? From the Collaborative International Dictionary of
English:
Distill \Dis*till"\, v. t.
[...]
5. to extract out and present the essence of; to shorten and
refine; to present the essential elements of; [...]
-- Niklas Matthies
Re: Ivy RoundUp Repository - feedback requested
Posted by Matt Benson <gu...@yahoo.com>.
--- Gilles Scokart <gs...@gmail.com> wrote:
> A few more feed back:
>
> 1. I like the fact that the ivy.xml and build.xml
> contains a license
> boilerplate. I think it should a rule, and that the
> AL should be
> applied systematically for all meta-data info stored
> in ivy-rep
> 2. As I said before, I don't like the term 'build'.
> Something like
> 'download', 'install', 'get', etc... would be much
> better.
To chime in here, I'd like to point out that in some
extreme hypothetical case a project might release in
source form only. In this case a build _would_ take
place. I'd like to see the RoundUp concept be
reusable within an organization, where the build/get
instructions might be to pull a particular svn tag,
then execute an Ant build. So maybe "build" assumes
too much, but we could explore some middleground that
simply implies that some processing must take place...
"prepare"?
-Matt
> 3. Signatures should be considered, and all
> meta-data files should be signed.
> 4. You should consider versioning of the meta-data.
> ivyroundup will
> be a downstream distributor, you can make errors
> that you will have to
> fix, without asking a new release from your upstream
> distributor, and
> still allowing build reproduceability.
> (by build reproduceability, I mean that the user
> must be able to
> define with high precision what he want to change).
> 5. A dependency repository needs to :
> - Notify users in case of a new version of a
> module that he use.
> - Provides release notes links.
> 6. You might encounter hosting issues
>
>
> On 15/04/2008, Archie Cobbs <ar...@dellroad.org>
> wrote:
> > Hello fellow Ivy users,
> >
> > I'd like to announce a new little project I've
> started, and ask for your
> > feedback (and help, if interested).
> >
> > This project has two basic parts...
> >
> > 1. *Builder
>
Resolver*<http://ivyroundup.googlecode.com/svn/wiki/files/builder.html>:
> > a new Ivy resolver that accesses ivy files and
> "build instructions" from an
> > online "builder" repository. "Builder"
> repositories contain ivy.xml files
> > but no artifacts. To get the artifacts, the
> build instructions are
> > downloaded from the repository and executed
> locally. These instructions
> > specify additional resource(s) to download and
> how to build the artifacts
> > from them, for example, by downloading a
> project's original distribution
> > archive directly from their web site and
> extracting the desired artifacts.
> > 2. *Ivy RoundUp Repository*
> <http://ivyroundup.googlecode.com/>: an
> > online, open-source community "Builder"
> repository for all Ivy users.
> >
> > Please click the links for more info and
> documentation.
> >
> > I am lobbying to get the builder resolver added
> into Ivy itself; right now
> > it's still in patch form (you can download a
> pre-built ivy.jar from the
> > project website).
> >
> > Some motivations for starting this project:
> >
> > 1. Ivyrep is no longer maintained, but we need
> a decent community Ivy
> > repository that everyone can share
> > 2. Hosting hundreds of large files that are
> just copies of the same
> > files available elsewhere is expensive and
> redundant, so let's avoid doing
> > that
> > 3. 99% of projects out there do not publish
> ivy.xml files, so we need
> > a community project that focuses on developing
> and maintaining them
> > 4. To get the most out of Ivy, there needs to
> be a consistent set of
> > guidelines for creating ivy.xml files: how to
> choose organization names,
> > philosophy for defining configurations, etc. A
> community project supported
> > by Ivy users can provide this.
> >
> > What I want to do is gauge interest in this idea
> and ask for any volunteers
> > who'd like to start adding and maintaining
> meta-data for their favorite
> > projects. The Ivy RoundUp repository is online
> now, though only as a
> > proof-of-concept (it only contains a few modules
> so far). Take a look and
> > you should be able to get the general idea:
> > http://ivyroundup.googlecode.com/
> >
> > In the worst case, if nobody else is interested,
> I will just use this for
> > myself -- it's already working better than what I
> was doing (i.e., checking
> > in giant ZIP files into Subversion and creating a
> project for every one to
> > publish into our private Ivy repository), and in
> any case the work of
> > setting it up is already done. Note also anyone
> could create their own
> > private builder repository using this project as
> well.
> >
> > In the best case, we'll put together a piece of
> infrastructure that all Ivy
> > users can really benefit from.
> >
> > Let me know what you think.
> >
> > Thanks,
> > -Archie
> >
> >
> > --
> > Archie L. Cobbs
> >
>
>
> --
> Gilles Scokart
>
____________________________________________________________________________________
Be a better friend, newshound, and
know-it-all with Yahoo! Mobile. Try it now. http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ
Re: Ivy RoundUp Repository - feedback requested
Posted by Gilles Scokart <gs...@gmail.com>.
A few more feed back:
1. I like the fact that the ivy.xml and build.xml contains a license
boilerplate. I think it should a rule, and that the AL should be
applied systematically for all meta-data info stored in ivy-rep
2. As I said before, I don't like the term 'build'. Something like
'download', 'install', 'get', etc... would be much better.
3. Signatures should be considered, and all meta-data files should be signed.
4. You should consider versioning of the meta-data. ivyroundup will
be a downstream distributor, you can make errors that you will have to
fix, without asking a new release from your upstream distributor, and
still allowing build reproduceability.
(by build reproduceability, I mean that the user must be able to
define with high precision what he want to change).
5. A dependency repository needs to :
- Notify users in case of a new version of a module that he use.
- Provides release notes links.
6. You might encounter hosting issues
On 15/04/2008, Archie Cobbs <ar...@dellroad.org> wrote:
> Hello fellow Ivy users,
>
> I'd like to announce a new little project I've started, and ask for your
> feedback (and help, if interested).
>
> This project has two basic parts...
>
> 1. *Builder Resolver*<http://ivyroundup.googlecode.com/svn/wiki/files/builder.html>:
> a new Ivy resolver that accesses ivy files and "build instructions" from an
> online "builder" repository. "Builder" repositories contain ivy.xml files
> but no artifacts. To get the artifacts, the build instructions are
> downloaded from the repository and executed locally. These instructions
> specify additional resource(s) to download and how to build the artifacts
> from them, for example, by downloading a project's original distribution
> archive directly from their web site and extracting the desired artifacts.
> 2. *Ivy RoundUp Repository* <http://ivyroundup.googlecode.com/>: an
> online, open-source community "Builder" repository for all Ivy users.
>
> Please click the links for more info and documentation.
>
> I am lobbying to get the builder resolver added into Ivy itself; right now
> it's still in patch form (you can download a pre-built ivy.jar from the
> project website).
>
> Some motivations for starting this project:
>
> 1. Ivyrep is no longer maintained, but we need a decent community Ivy
> repository that everyone can share
> 2. Hosting hundreds of large files that are just copies of the same
> files available elsewhere is expensive and redundant, so let's avoid doing
> that
> 3. 99% of projects out there do not publish ivy.xml files, so we need
> a community project that focuses on developing and maintaining them
> 4. To get the most out of Ivy, there needs to be a consistent set of
> guidelines for creating ivy.xml files: how to choose organization names,
> philosophy for defining configurations, etc. A community project supported
> by Ivy users can provide this.
>
> What I want to do is gauge interest in this idea and ask for any volunteers
> who'd like to start adding and maintaining meta-data for their favorite
> projects. The Ivy RoundUp repository is online now, though only as a
> proof-of-concept (it only contains a few modules so far). Take a look and
> you should be able to get the general idea:
> http://ivyroundup.googlecode.com/
>
> In the worst case, if nobody else is interested, I will just use this for
> myself -- it's already working better than what I was doing (i.e., checking
> in giant ZIP files into Subversion and creating a project for every one to
> publish into our private Ivy repository), and in any case the work of
> setting it up is already done. Note also anyone could create their own
> private builder repository using this project as well.
>
> In the best case, we'll put together a piece of infrastructure that all Ivy
> users can really benefit from.
>
> Let me know what you think.
>
> Thanks,
> -Archie
>
>
> --
> Archie L. Cobbs
>
--
Gilles Scokart
Re: Ivy RoundUp Repository - feedback requested
Posted by Archie Cobbs <ar...@dellroad.org>.
On Tue, Apr 15, 2008 at 10:48 AM, Archie Cobbs <ar...@dellroad.org> wrote:
> A way to address both of your comments would be to allow configuration of
> a "resource URL override" pattern: when configured, the resource URLs in the
> build instructions would be ignored, and instead all resources would be
> downloaded using the override URL pattern to create the URL. Then you could
> keep your own private mirror. Hmm, that should be easy to add.
>
>
I've added this capability using a new "resourceURL" attribute and updated
the patch, ivy.jar, and documentation on the website.
Thanks,
-Archie
--
Archie L. Cobbs
Re: Ivy RoundUp Repository - feedback requested
Posted by Archie Cobbs <ar...@dellroad.org>.
On Tue, Apr 15, 2008 at 10:22 AM, jonathan doklovic <jd...@ibsys.com>
wrote:
> I'm not sure I'm a fan of the build instructions though. It seems that
> pulling artifacts from the creators websites has some problems. Off the
> top of my head... it could add a lot of time to resolve since a lot of
> OS sites are slow. (including sourceforge). Also, as time goes by,
> sites have a tendency to archive artifacts and sometimes even remove
> them altogether, which would then in turn break the build instructions
> in either case. Just seems like it might be more maintenance than it's
> worth.
>
>
My thinking is that generally you'd try to avoid "building". Instead, use
Ivy's cache, and/or the builder resolver's resource cache. So if the
download is slow, at least it's only slow the first time... this helps but
doesn't eliminate the problem.
As for the problem of the original distributions disappearing, that is also
a valid point. It can be treated as an orthogonal problem however. For
example, there's no reason the build instructions can't point to resources
hosted on some "permanent mirror" site set up just for that purpose, instead
of the original projects' site, etc. Perhaps that can be a follow-on project
to this one to add robustness.
A way to address both of your comments would be to allow configuration of a
"resource URL override" pattern: when configured, the resource URLs in the
build instructions would be ignored, and instead all resources would be
downloaded using the override URL pattern to create the URL. Then you could
keep your own private mirror. Hmm, that should be easy to add.
Thanks for the comments.
-Archie
--
Archie L. Cobbs
Re: Ivy RoundUp Repository - feedback requested
Posted by Xavier Hanin <xa...@gmail.com>.
On Wed, Apr 16, 2008 at 4:42 PM, Archie Cobbs <ar...@dellroad.org> wrote:
> On Wed, Apr 16, 2008 at 2:41 AM, Xavier Hanin <xa...@gmail.com>
> wrote:
>
> > That being said, I see a problem with hosting the repository in google
> > code
> > svn repo: access seems to be very slow sometimes, and subversion doesn't
> > help. Maybe it would be interesting to have a public file server
> launching
> > an svn up in a cron? I have a small box which I use for
> xoocode.orghosting,
> > I could set this up if you think it's interesting.
>
>
> Yes, that would be easy. If you checkout the project, you'll see that when
> you run "ant" it builds the entire repository as a directory tree under
> repo/. Then, upon successful review, the developer would commit that repo/
> subtree into SVN and it automatically becomes available via HTTP from
> Google
> SVN.
>
> Of course there's no reason you couldn't take repo/ and copy it to some
> faster web server if you wanted, e.g. using svn update as you suggest.
>
> I haven't experienced slowness with Google SVN, and the files being
> downloaded are small, but on the other hand I haven't done any performance
> testing.
This morning the repository was very slow, with even bad responses (empty
pages). But it was the first time I experienced this... So maybe it's not
worth trying to use something else ATM.
Xavier
>
>
> Sure, let's try it. Here's the URL to checkout:
> http://ivyroundup.googlecode.com/svn/trunk/repo
>
> -Archie
>
> --
> Archie L. Cobbs
>
--
Xavier Hanin - Independent Java Consultant
http://xhab.blogspot.com/
http://ant.apache.org/ivy/
http://www.xoocode.org/
Re: Ivy RoundUp Repository - feedback requested
Posted by Archie Cobbs <ar...@dellroad.org>.
On Wed, Apr 16, 2008 at 2:41 AM, Xavier Hanin <xa...@gmail.com>
wrote:
> That being said, I see a problem with hosting the repository in google
> code
> svn repo: access seems to be very slow sometimes, and subversion doesn't
> help. Maybe it would be interesting to have a public file server launching
> an svn up in a cron? I have a small box which I use for xoocode.orghosting,
> I could set this up if you think it's interesting.
Yes, that would be easy. If you checkout the project, you'll see that when
you run "ant" it builds the entire repository as a directory tree under
repo/. Then, upon successful review, the developer would commit that repo/
subtree into SVN and it automatically becomes available via HTTP from Google
SVN.
Of course there's no reason you couldn't take repo/ and copy it to some
faster web server if you wanted, e.g. using svn update as you suggest.
I haven't experienced slowness with Google SVN, and the files being
downloaded are small, but on the other hand I haven't done any performance
testing.
Sure, let's try it. Here's the URL to checkout:
http://ivyroundup.googlecode.com/svn/trunk/repo
-Archie
--
Archie L. Cobbs
Re: Ivy RoundUp Repository - feedback requested
Posted by Xavier Hanin <xa...@gmail.com>.
On Tue, Apr 15, 2008 at 5:22 PM, jonathan doklovic <jd...@ibsys.com>
wrote:
> Interesting idea.
>
> I really think there is a need for a stable public repo as well as some
> standards for conf naming and such.
>
> I'm not sure I'm a fan of the build instructions though. It seems that
> pulling artifacts from the creators websites has some problems. Off the
> top of my head... it could add a lot of time to resolve since a lot of
> OS sites are slow. (including sourceforge). Also, as time goes by,
> sites have a tendency to archive artifacts and sometimes even remove
> them altogether, which would then in turn break the build instructions
> in either case. Just seems like it might be more maintenance than it's
> worth.
>
> In my opinion, disk space is cheap and it would be more reliable to
> store copies of the artifacts in the public repo.
I agree with your arguments, but I do think storing build instructions is a
very good idea. By storing the build instructions, you get an easier import
of new versions from the project: let's say you have already written build
instructions for foo 1.1, when foo 1.2 is released, there is good chance
that your build instructions will only require minor adjustments. If
downloading, unzipping, repackaging sources and javadocs were done by hand,
you would have to reproduce the same steps each time.
Now I see this approach as an intermediary step: once you have a repository
with build instructions, it's super easy to setup a repository with actual
artifacts, built automatically from the build instructions. And if enough
people use the builder approach, I guess there will be enough interest to
get money to setup this repository.
That being said, I see a problem with hosting the repository in google code
svn repo: access seems to be very slow sometimes, and subversion doesn't
help. Maybe it would be interesting to have a public file server launching
an svn up in a cron? I have a small box which I use for xoocode.org hosting,
I could set this up if you think it's interesting.
Xavier
>
>
> Cool work though.
>
> - Jonathan
>
>
> On Tue, 2008-04-15 at 10:06 -0500, Archie Cobbs wrote:
> > Hello fellow Ivy users,
> >
> > I'd like to announce a new little project I've started, and ask for your
> > feedback (and help, if interested).
> >
> > This project has two basic parts...
> >
> > 1. *Builder Resolver*<
> http://ivyroundup.googlecode.com/svn/wiki/files/builder.html>:
> > a new Ivy resolver that accesses ivy files and "build instructions"
> from an
> > online "builder" repository. "Builder" repositories contain ivy.xml
> files
> > but no artifacts. To get the artifacts, the build instructions are
> > downloaded from the repository and executed locally. These
> instructions
> > specify additional resource(s) to download and how to build the
> artifacts
> > from them, for example, by downloading a project's original
> distribution
> > archive directly from their web site and extracting the desired
> artifacts.
> > 2. *Ivy RoundUp Repository* <http://ivyroundup.googlecode.com/>: an
> > online, open-source community "Builder" repository for all Ivy users.
> >
> > Please click the links for more info and documentation.
> >
> > I am lobbying to get the builder resolver added into Ivy itself; right
> now
> > it's still in patch form (you can download a pre-built ivy.jar from the
> > project website).
> >
> > Some motivations for starting this project:
> >
> > 1. Ivyrep is no longer maintained, but we need a decent community Ivy
> > repository that everyone can share
> > 2. Hosting hundreds of large files that are just copies of the same
> > files available elsewhere is expensive and redundant, so let's avoid
> doing
> > that
> > 3. 99% of projects out there do not publish ivy.xml files, so we need
> > a community project that focuses on developing and maintaining them
> > 4. To get the most out of Ivy, there needs to be a consistent set of
> > guidelines for creating ivy.xml files: how to choose organization
> names,
> > philosophy for defining configurations, etc. A community project
> supported
> > by Ivy users can provide this.
> >
> > What I want to do is gauge interest in this idea and ask for any
> volunteers
> > who'd like to start adding and maintaining meta-data for their favorite
> > projects. The Ivy RoundUp repository is online now, though only as a
> > proof-of-concept (it only contains a few modules so far). Take a look
> and
> > you should be able to get the general idea:
> > http://ivyroundup.googlecode.com/
> >
> > In the worst case, if nobody else is interested, I will just use this
> for
> > myself -- it's already working better than what I was doing (i.e.,
> checking
> > in giant ZIP files into Subversion and creating a project for every one
> to
> > publish into our private Ivy repository), and in any case the work of
> > setting it up is already done. Note also anyone could create their own
> > private builder repository using this project as well.
> >
> > In the best case, we'll put together a piece of infrastructure that all
> Ivy
> > users can really benefit from.
> >
> > Let me know what you think.
> >
> > Thanks,
> > -Archie
> >
>
--
Xavier Hanin - Independent Java Consultant
http://xhab.blogspot.com/
http://ant.apache.org/ivy/
http://www.xoocode.org/
Re: Ivy RoundUp Repository - feedback requested
Posted by jonathan doklovic <jd...@ibsys.com>.
Interesting idea.
I really think there is a need for a stable public repo as well as some
standards for conf naming and such.
I'm not sure I'm a fan of the build instructions though. It seems that
pulling artifacts from the creators websites has some problems. Off the
top of my head... it could add a lot of time to resolve since a lot of
OS sites are slow. (including sourceforge). Also, as time goes by,
sites have a tendency to archive artifacts and sometimes even remove
them altogether, which would then in turn break the build instructions
in either case. Just seems like it might be more maintenance than it's
worth.
In my opinion, disk space is cheap and it would be more reliable to
store copies of the artifacts in the public repo.
Cool work though.
- Jonathan
On Tue, 2008-04-15 at 10:06 -0500, Archie Cobbs wrote:
> Hello fellow Ivy users,
>
> I'd like to announce a new little project I've started, and ask for your
> feedback (and help, if interested).
>
> This project has two basic parts...
>
> 1. *Builder Resolver*<http://ivyroundup.googlecode.com/svn/wiki/files/builder.html>:
> a new Ivy resolver that accesses ivy files and "build instructions" from an
> online "builder" repository. "Builder" repositories contain ivy.xml files
> but no artifacts. To get the artifacts, the build instructions are
> downloaded from the repository and executed locally. These instructions
> specify additional resource(s) to download and how to build the artifacts
> from them, for example, by downloading a project's original distribution
> archive directly from their web site and extracting the desired artifacts.
> 2. *Ivy RoundUp Repository* <http://ivyroundup.googlecode.com/>: an
> online, open-source community "Builder" repository for all Ivy users.
>
> Please click the links for more info and documentation.
>
> I am lobbying to get the builder resolver added into Ivy itself; right now
> it's still in patch form (you can download a pre-built ivy.jar from the
> project website).
>
> Some motivations for starting this project:
>
> 1. Ivyrep is no longer maintained, but we need a decent community Ivy
> repository that everyone can share
> 2. Hosting hundreds of large files that are just copies of the same
> files available elsewhere is expensive and redundant, so let's avoid doing
> that
> 3. 99% of projects out there do not publish ivy.xml files, so we need
> a community project that focuses on developing and maintaining them
> 4. To get the most out of Ivy, there needs to be a consistent set of
> guidelines for creating ivy.xml files: how to choose organization names,
> philosophy for defining configurations, etc. A community project supported
> by Ivy users can provide this.
>
> What I want to do is gauge interest in this idea and ask for any volunteers
> who'd like to start adding and maintaining meta-data for their favorite
> projects. The Ivy RoundUp repository is online now, though only as a
> proof-of-concept (it only contains a few modules so far). Take a look and
> you should be able to get the general idea:
> http://ivyroundup.googlecode.com/
>
> In the worst case, if nobody else is interested, I will just use this for
> myself -- it's already working better than what I was doing (i.e., checking
> in giant ZIP files into Subversion and creating a project for every one to
> publish into our private Ivy repository), and in any case the work of
> setting it up is already done. Note also anyone could create their own
> private builder repository using this project as well.
>
> In the best case, we'll put together a piece of infrastructure that all Ivy
> users can really benefit from.
>
> Let me know what you think.
>
> Thanks,
> -Archie
>
Re: Ivy RoundUp Repository - feedback requested
Posted by Xavier Hanin <xa...@gmail.com>.
On Fri, Apr 18, 2008 at 6:56 PM, Xavier Hanin <xa...@gmail.com>
wrote:
> On Fri, Apr 18, 2008 at 5:01 PM, Archie Cobbs <ar...@dellroad.org> wrote:
>
> > On Thu, Apr 17, 2008 at 11:22 PM, Chris Hane <ch...@gmail.com>
> > wrote:
> >
> > > I think have a meta repository is a great step. I have been following
> > ivy
> > > for a while. I have on my todo list to replace (enhance) our project
> > build
> > > files to use Ivy. However, I have been dragging my feet because I
> > don't
> > > want to build all of the ivy files that I will need. I think having a
> > > repository where ivy files can be shared is great - it's just icing
> > that we
> > > can have a plugin for ivy to do a lot of the repacking work.
> > >
> >
> > Your experience sounds very similar to mine and is exactly what
> > motivated
> > this project in the first place...
> >
> >
> > > I've noticed that you put up an initial list of ivy files. Do you
> > have a
> > > process for adding more? Submit patches to the issue tracker?
> > >
> >
> > Right now I've just thrown a few modules in there as sort-of a
> > proof-of-concept. Here's what I would like to happen next:
> >
> > 1. Get some additional contributors to join the Ivy RoundUp project.
> > I'm envisioning a fairly open project, meaning lots of "fine grained"
> > committers, or perhaps a small team of committers plus a larger pool
> > of
> > "maintainers" who are assigned responsibility for each module. The
> > goal here
> > is to allow people to focus on the modules they are interested in.
> > This is
> > analogous to how FreeBSD's ports system works. The issue tracker is a
> > good
> > vehicle for tracking requested additions and changes.
> > 2. Decide what to do about the stuff in the maven2 repository. We have
> > the option here of building an automated tool that would "import"
> > everything
> > in maven2, creating the ivy.xml equivalents of the POM, etc. for each
> > module
> > on the fly. These ivy.xml files would form a starting point for
> > further
> > refinements over time.
> > 3. Flesh out the
> > ModuleMaintainerGuidelines<
> > http://code.google.com/p/ivyroundup/wiki/ModuleMaintainerGuidelines
> > >based
> > on the collective wisdom on this list.
> > 4. Establish a place for discussion and decision-making about module
> > naming, configurations, dependency mapping, etc. Add some new modules
> > to
> > stimulate thinking.
> >
> > For #1, for now anybody who wants to join as a committer is welcome..
> > get in
> > on the ground floor! :-)
>
> I step up, even though I'm not sure to get much time to involve.
>
>
> >
> > For #2, this is an interesting discussion. I am willing to write the
> > code,
> > but I need the help of this list to understand what is the best
> > approach.
>
> Importing stuff from maven 2 is interesting because there's already a lot
> of modules. But sometimes the quality is really not there, and the risk of a
> massive import is to inherit the low level of quality. So I think we need to
> be very careful with the import, and make import only when necessary at
> least to start. A lot can be necessary pretty quickly (some projects bring a
> lot of dependencies). IMO, what's very important to start having a good repo
> is to have some tools to check the consistency. I think we need at least to
> ensure that each module can be resolved alone with Ivy used with an empty
> cache. This check is pretty easy to implement, and will save a lot of
> mistakes. We could even put the xml resolution report in the repository: it
> could be useful to have transitive dependencies information.
>
> If we agree on package like naming convention for organization, another
> interesting check is to check the organization name compared to the package
> name (for java projects).
>
> IMO implementing these checks is the most important to ensure a high level
> of quality from the beginning. With these tools, it will be a lot safer to
> provide commit rights to a large number of people. The problem is that
> writing these tools require some time.
>
Speaking about tooling, some of you may be aware sometime ago I participated
to a project called worldofjava, which is now dead... This project provided
access to source and javadocs of a large number of projects. For those that
were not available in maven repo, or without sources or javadocs, we built a
tool which was able to scan some web sites to find new versions
(sourceforge, ASF, objectweb, ...), download the archives, analyze the
content and try find interesting bits in the archive. This analysis
generates a descriptor (pretty much like your build, but more descriptor
oriented than task oriented) that we edited manually when the automatic
analysis was not good enough.
I'm speaking about this because the code is now open source (meaning only
that anyone can reuse it, there's no community behind). So it might be
interesting to reuse some parts of this tool. Here's the source of the
analyzer:
http://woj.svn.sourceforge.net/viewvc/woj/woj-tools/src/java/org/jayasoft/woj/tools/analyzer/DLAnalyzer.java?revision=17&view=markup
BTW, jayasoft is still the owner of the repository used by WOJ, and I'm the
co owner of Jayasoft, so I can discuss with the other owner to see if we can
open source the repository too. It doesn't contain any Ivy files (maybe one
or two), but it already contains a lot of cleanly extracted sources,
javadocs, artifacts, and descriptors to extract them from the original
archives. Don't know if you think it would be interesting...
Xavier
>
>
> Xavier
>
>
> >
> > For #3, I'd ask for anyone interested to add their thoughts and update
> > the
> > wiki.
> >
> > For #4, let's continue to use this mailing list for the time being. When
> > the
> > email traffic becomes more substantial we'll create a separate list.
> >
> > I'm looking forward to getting this up and running. I think it can
> > become a
> > great resource for all Ivy users.
> >
> > -Archie
> >
> > --
> > Archie L. Cobbs
> >
>
>
>
> --
> Xavier Hanin - Independent Java Consultant
> http://xhab.blogspot.com/
> http://ant.apache.org/ivy/
> http://www.xoocode.org/
>
--
Xavier Hanin - Independent Java Consultant
http://xhab.blogspot.com/
http://ant.apache.org/ivy/
http://www.xoocode.org/
Re: Ivy RoundUp Repository - feedback requested
Posted by Archie Cobbs <ar...@dellroad.org>.
On Fri, Apr 18, 2008 at 11:56 AM, Xavier Hanin <xa...@gmail.com>
wrote:
> > For #1, for now anybody who wants to join as a committer is welcome..
> get
> > in on the ground floor! :-)
>
> I step up, even though I'm not sure to get much time to involve.
>
Please email me your google ID and I'll add you as a member.
> > For #2, this is an interesting discussion. I am willing to write the
> code,
> > but I need the help of this list to understand what is the best
> approach.
>
> Importing stuff from maven 2 is interesting because there's already a lot
> of
> modules. But sometimes the quality is really not there, and the risk of a
> massive import is to inherit the low level of quality. So I think we need
> to
> be very careful with the import, and make import only when necessary at
> least to start. A lot can be necessary pretty quickly (some projects bring
> a
> lot of dependencies). IMO, what's very important to start having a good
> repo
> is to have some tools to check the consistency. I think we need at least
> to
> ensure that each module can be resolved alone with Ivy used with an empty
> cache. This check is pretty easy to implement, and will save a lot of
> mistakes. We could even put the xml resolution report in the repository:
> it
> could be useful to have transitive dependencies information.
>
> If we agree on package like naming convention for organization, another
> interesting check is to check the organization name compared to the
> package
> name (for java projects).
>
> IMO implementing these checks is the most important to ensure a high level
> of quality from the beginning. With these tools, it will be a lot safer to
> provide commit rights to a large number of people. The problem is that
> writing these tools require some time.
>
I agree that quality should take priority over quantity. Quantity will
happen naturally over time... but quality won't! It only happens through
deliberate effort, good automated tools, etc.
In other words, as more people use the repository, they will run across
modules that are missing... at that time we will add the missing module,
possibly with help from the person who needed it. So the repository grows in
line with the needs and interests of its user base.
I will work on implementing some further automated checks.
-Archie
--
Archie L. Cobbs
Re: Ivy RoundUp Repository - feedback requested
Posted by Xavier Hanin <xa...@gmail.com>.
On Fri, Apr 18, 2008 at 5:01 PM, Archie Cobbs <ar...@dellroad.org> wrote:
> On Thu, Apr 17, 2008 at 11:22 PM, Chris Hane <ch...@gmail.com> wrote:
>
> > I think have a meta repository is a great step. I have been following
> ivy
> > for a while. I have on my todo list to replace (enhance) our project
> build
> > files to use Ivy. However, I have been dragging my feet because I don't
> > want to build all of the ivy files that I will need. I think having a
> > repository where ivy files can be shared is great - it's just icing that
> we
> > can have a plugin for ivy to do a lot of the repacking work.
> >
>
> Your experience sounds very similar to mine and is exactly what motivated
> this project in the first place...
>
>
> > I've noticed that you put up an initial list of ivy files. Do you have
> a
> > process for adding more? Submit patches to the issue tracker?
> >
>
> Right now I've just thrown a few modules in there as sort-of a
> proof-of-concept. Here's what I would like to happen next:
>
> 1. Get some additional contributors to join the Ivy RoundUp project.
> I'm envisioning a fairly open project, meaning lots of "fine grained"
> committers, or perhaps a small team of committers plus a larger pool of
> "maintainers" who are assigned responsibility for each module. The goal
> here
> is to allow people to focus on the modules they are interested in. This
> is
> analogous to how FreeBSD's ports system works. The issue tracker is a
> good
> vehicle for tracking requested additions and changes.
> 2. Decide what to do about the stuff in the maven2 repository. We have
> the option here of building an automated tool that would "import"
> everything
> in maven2, creating the ivy.xml equivalents of the POM, etc. for each
> module
> on the fly. These ivy.xml files would form a starting point for further
> refinements over time.
> 3. Flesh out the
> ModuleMaintainerGuidelines<
> http://code.google.com/p/ivyroundup/wiki/ModuleMaintainerGuidelines>based
> on the collective wisdom on this list.
> 4. Establish a place for discussion and decision-making about module
> naming, configurations, dependency mapping, etc. Add some new modules to
> stimulate thinking.
>
> For #1, for now anybody who wants to join as a committer is welcome.. get
> in
> on the ground floor! :-)
I step up, even though I'm not sure to get much time to involve.
>
> For #2, this is an interesting discussion. I am willing to write the code,
> but I need the help of this list to understand what is the best approach.
Importing stuff from maven 2 is interesting because there's already a lot of
modules. But sometimes the quality is really not there, and the risk of a
massive import is to inherit the low level of quality. So I think we need to
be very careful with the import, and make import only when necessary at
least to start. A lot can be necessary pretty quickly (some projects bring a
lot of dependencies). IMO, what's very important to start having a good repo
is to have some tools to check the consistency. I think we need at least to
ensure that each module can be resolved alone with Ivy used with an empty
cache. This check is pretty easy to implement, and will save a lot of
mistakes. We could even put the xml resolution report in the repository: it
could be useful to have transitive dependencies information.
If we agree on package like naming convention for organization, another
interesting check is to check the organization name compared to the package
name (for java projects).
IMO implementing these checks is the most important to ensure a high level
of quality from the beginning. With these tools, it will be a lot safer to
provide commit rights to a large number of people. The problem is that
writing these tools require some time.
Xavier
>
> For #3, I'd ask for anyone interested to add their thoughts and update the
> wiki.
>
> For #4, let's continue to use this mailing list for the time being. When
> the
> email traffic becomes more substantial we'll create a separate list.
>
> I'm looking forward to getting this up and running. I think it can become
> a
> great resource for all Ivy users.
>
> -Archie
>
> --
> Archie L. Cobbs
>
--
Xavier Hanin - Independent Java Consultant
http://xhab.blogspot.com/
http://ant.apache.org/ivy/
http://www.xoocode.org/
Re: Ivy RoundUp Repository - feedback requested
Posted by Archie Cobbs <ar...@dellroad.org>.
On Thu, Apr 17, 2008 at 11:22 PM, Chris Hane <ch...@gmail.com> wrote:
> I think have a meta repository is a great step. I have been following ivy
> for a while. I have on my todo list to replace (enhance) our project build
> files to use Ivy. However, I have been dragging my feet because I don't
> want to build all of the ivy files that I will need. I think having a
> repository where ivy files can be shared is great - it's just icing that we
> can have a plugin for ivy to do a lot of the repacking work.
>
Your experience sounds very similar to mine and is exactly what motivated
this project in the first place...
> I've noticed that you put up an initial list of ivy files. Do you have a
> process for adding more? Submit patches to the issue tracker?
>
Right now I've just thrown a few modules in there as sort-of a
proof-of-concept. Here's what I would like to happen next:
1. Get some additional contributors to join the Ivy RoundUp project.
I'm envisioning a fairly open project, meaning lots of "fine grained"
committers, or perhaps a small team of committers plus a larger pool of
"maintainers" who are assigned responsibility for each module. The goal here
is to allow people to focus on the modules they are interested in. This is
analogous to how FreeBSD's ports system works. The issue tracker is a good
vehicle for tracking requested additions and changes.
2. Decide what to do about the stuff in the maven2 repository. We have
the option here of building an automated tool that would "import" everything
in maven2, creating the ivy.xml equivalents of the POM, etc. for each module
on the fly. These ivy.xml files would form a starting point for further
refinements over time.
3. Flesh out the
ModuleMaintainerGuidelines<http://code.google.com/p/ivyroundup/wiki/ModuleMaintainerGuidelines>based
on the collective wisdom on this list.
4. Establish a place for discussion and decision-making about module
naming, configurations, dependency mapping, etc. Add some new modules to
stimulate thinking.
For #1, for now anybody who wants to join as a committer is welcome.. get in
on the ground floor! :-)
For #2, this is an interesting discussion. I am willing to write the code,
but I need the help of this list to understand what is the best approach.
For #3, I'd ask for anyone interested to add their thoughts and update the
wiki.
For #4, let's continue to use this mailing list for the time being. When the
email traffic becomes more substantial we'll create a separate list.
I'm looking forward to getting this up and running. I think it can become a
great resource for all Ivy users.
-Archie
--
Archie L. Cobbs
Re: Ivy RoundUp Repository - feedback requested
Posted by Chris Hane <ch...@gmail.com>.
>> Also, a general request - does anyone have a hibernate ivy file they would
>> like to contribute?
>
> You have some here:
> https://opensvn.csie.org/ivyrepsandbox/hibernate/hibernate/
>
> Xavier
>
Thanks for the link.
Chris....
Re: Ivy RoundUp Repository - feedback requested
Posted by Archie Cobbs <ar...@dellroad.org>.
On Fri, Apr 18, 2008 at 11:46 AM, Xavier Hanin <xa...@gmail.com>
wrote:
> > More generally, is there a pool of "high quality" ivy.xml files out
> there
> > somewhere that we should import into RoundUp?
>
> The only high quality ivy files I'm aware of are in IvyRep. And even those
> are not of very high quality considering the naming conventions used. I
> think they are a good source of inspiration, but that's pretty much all.
I will work on copying them to Ivy RoundUp... but using java package naming
conventions (subject to further review).
> Thinking out loud about quality, I think it will be difficult to get a
> good
> quality from scratch. So maybe the repository should have versions as you
> already suggested: people who want to use the latest updates use the
> trunk,
> while others use 0.x versions to get something more stable. Once we'll
> have
> a large and consistent enough repo, we could go to 1.0 and never break
> backward compatiblity from there. Then a vote mechanism to ensure new
> metadata is still consistent and good enough would be very interesting.
>
Yes I like that approach.
-Archie
--
Archie L. Cobbs
Re: Ivy RoundUp Repository - feedback requested
Posted by Xavier Hanin <xa...@gmail.com>.
On Fri, Apr 18, 2008 at 6:06 PM, Archie Cobbs <ar...@dellroad.org> wrote:
> On Fri, Apr 18, 2008 at 5:13 AM, Xavier Hanin <xa...@gmail.com>
> wrote:
>
> > > Also, a general request - does anyone have a hibernate ivy file they
> > would
> > > like to contribute?
> >
> > You have some here:
> > https://opensvn.csie.org/ivyrepsandbox/hibernate/hibernate/
> >
> >
> Hey Xavier, is this "ivyrepsandbox" your project? What is its purpose?
It's as old as ivyrep. There's a link and some explanation here:
http://www.jaya.free.fr/ivyrep/
People willing to contribute files over there used to ask for an account.
Unfortunately I don't think I still receive the e-mails, so nobody can get
new accounts (except by asking directly). I don't know if people with
accounts still commits things over there.
>
>
> Would it make sense to import all these files into Ivy Round Up? Under
> what
> license are they?
No license, people contributed them with no specific agreement...
>
>
> More generally, is there a pool of "high quality" ivy.xml files out there
> somewhere that we should import into RoundUp?
The only high quality ivy files I'm aware of are in IvyRep. And even those
are not of very high quality considering the naming conventions used. I
think they are a good source of inspiration, but that's pretty much all.
Thinking out loud about quality, I think it will be difficult to get a good
quality from scratch. So maybe the repository should have versions as you
already suggested: people who want to use the latest updates use the trunk,
while others use 0.x versions to get something more stable. Once we'll have
a large and consistent enough repo, we could go to 1.0 and never break
backward compatiblity from there. Then a vote mechanism to ensure new
metadata is still consistent and good enough would be very interesting.
Xavier
>
>
> Thanks,
> -Archie
>
> --
> Archie L. Cobbs
>
--
Xavier Hanin - Independent Java Consultant
http://xhab.blogspot.com/
http://ant.apache.org/ivy/
http://www.xoocode.org/
Re: Ivy RoundUp Repository - feedback requested
Posted by Archie Cobbs <ar...@dellroad.org>.
I think we should try to get them submitted under the Apache license, but
really any open-source license is OK because all we are doing is putting the
file up on a web site.
On Fri, Apr 18, 2008 at 1:45 PM, Chris Hane <ch...@gmail.com> wrote:
>
>
> Archie Cobbs wrote:
>
> > Hey Xavier, is this "ivyrepsandbox" your project? What is its purpose?
> >
> > Would it make sense to import all these files into Ivy Round Up? Under
> > what
> > license are they?
> >
> > More generally, is there a pool of "high quality" ivy.xml files out
> > there
> > somewhere that we should import into RoundUp?
> >
> >
> What will be the license requirements for submitting ivy.xml files?
>
> I've noticed that some projects are starting to create them (at least
> initial stabs). The specific projects I am thinking of are LGPL for the
> code. Not sure how that applies to a configuration file (but I'm not a
> lawyer)....
>
> Chris....
>
--
Archie L. Cobbs
Re: Ivy RoundUp Repository - feedback requested
Posted by Chris Hane <ch...@gmail.com>.
Archie Cobbs wrote:
> Hey Xavier, is this "ivyrepsandbox" your project? What is its purpose?
>
> Would it make sense to import all these files into Ivy Round Up? Under what
> license are they?
>
> More generally, is there a pool of "high quality" ivy.xml files out there
> somewhere that we should import into RoundUp?
>
What will be the license requirements for submitting ivy.xml files?
I've noticed that some projects are starting to create them (at least
initial stabs). The specific projects I am thinking of are LGPL for the
code. Not sure how that applies to a configuration file (but I'm not a
lawyer)....
Chris....
Re: Ivy RoundUp Repository - feedback requested
Posted by Archie Cobbs <ar...@dellroad.org>.
On Fri, Apr 18, 2008 at 5:13 AM, Xavier Hanin <xa...@gmail.com>
wrote:
> > Also, a general request - does anyone have a hibernate ivy file they
> would
> > like to contribute?
>
> You have some here:
> https://opensvn.csie.org/ivyrepsandbox/hibernate/hibernate/
>
>
Hey Xavier, is this "ivyrepsandbox" your project? What is its purpose?
Would it make sense to import all these files into Ivy Round Up? Under what
license are they?
More generally, is there a pool of "high quality" ivy.xml files out there
somewhere that we should import into RoundUp?
Thanks,
-Archie
--
Archie L. Cobbs
Re: Ivy RoundUp Repository - feedback requested
Posted by Xavier Hanin <xa...@gmail.com>.
On Fri, Apr 18, 2008 at 6:22 AM, Chris Hane <ch...@gmail.com> wrote:
> Archie - thanks!
>
> I think have a meta repository is a great step. I have been following ivy
> for a while. I have on my todo list to replace (enhance) our project build
> files to use Ivy. However, I have been dragging my feet because I don't
> want to build all of the ivy files that I will need. I think having a
> repository where ivy files can be shared is great - it's just icing that we
> can have a plugin for ivy to do a lot of the repacking work.
>
> I've noticed that you put up an initial list of ivy files. Do you have a
> process for adding more? Submit patches to the issue tracker?
>
> Also, a general request - does anyone have a hibernate ivy file they would
> like to contribute?
You have some here:
https://opensvn.csie.org/ivyrepsandbox/hibernate/hibernate/
Xavier
> That's always been the one I started with and have always stopped with it
> due to lack of time to figure everything out.
>
> Chris....
>
>
> Archie Cobbs wrote:
>
> > Hello fellow Ivy users,
> >
> > I'd like to announce a new little project I've started, and ask for your
> > feedback (and help, if interested).
> >
> > This project has two basic parts...
> >
> > 1. *Builder Resolver*<
> > http://ivyroundup.googlecode.com/svn/wiki/files/builder.html>:
> > a new Ivy resolver that accesses ivy files and "build instructions"
> > from an
> > online "builder" repository. "Builder" repositories contain ivy.xml
> > files
> > but no artifacts. To get the artifacts, the build instructions are
> > downloaded from the repository and executed locally. These
> > instructions
> > specify additional resource(s) to download and how to build the
> > artifacts
> > from them, for example, by downloading a project's original
> > distribution
> > archive directly from their web site and extracting the desired
> > artifacts.
> > 2. *Ivy RoundUp Repository* <http://ivyroundup.googlecode.com/>: an
> > online, open-source community "Builder" repository for all Ivy users.
> >
> > Please click the links for more info and documentation.
> >
> > I am lobbying to get the builder resolver added into Ivy itself; right
> > now
> > it's still in patch form (you can download a pre-built ivy.jar from the
> > project website).
> >
> > Some motivations for starting this project:
> >
> > 1. Ivyrep is no longer maintained, but we need a decent community Ivy
> > repository that everyone can share
> > 2. Hosting hundreds of large files that are just copies of the same
> > files available elsewhere is expensive and redundant, so let's avoid
> > doing
> > that
> > 3. 99% of projects out there do not publish ivy.xml files, so we need
> > a community project that focuses on developing and maintaining them
> > 4. To get the most out of Ivy, there needs to be a consistent set of
> > guidelines for creating ivy.xml files: how to choose organization
> > names,
> > philosophy for defining configurations, etc. A community project
> > supported
> > by Ivy users can provide this.
> >
> > What I want to do is gauge interest in this idea and ask for any
> > volunteers
> > who'd like to start adding and maintaining meta-data for their favorite
> > projects. The Ivy RoundUp repository is online now, though only as a
> > proof-of-concept (it only contains a few modules so far). Take a look
> > and
> > you should be able to get the general idea:
> > http://ivyroundup.googlecode.com/
> >
> > In the worst case, if nobody else is interested, I will just use this
> > for
> > myself -- it's already working better than what I was doing (i.e.,
> > checking
> > in giant ZIP files into Subversion and creating a project for every one
> > to
> > publish into our private Ivy repository), and in any case the work of
> > setting it up is already done. Note also anyone could create their own
> > private builder repository using this project as well.
> >
> > In the best case, we'll put together a piece of infrastructure that all
> > Ivy
> > users can really benefit from.
> >
> > Let me know what you think.
> >
> > Thanks,
> > -Archie
> >
> >
--
Xavier Hanin - Independent Java Consultant
http://xhab.blogspot.com/
http://ant.apache.org/ivy/
http://www.xoocode.org/
Re: Ivy RoundUp Repository - feedback requested
Posted by Chris Hane <ch...@gmail.com>.
Archie - thanks!
I think have a meta repository is a great step. I have been following ivy
for a while. I have on my todo list to replace (enhance) our project build
files to use Ivy. However, I have been dragging my feet because I don't
want to build all of the ivy files that I will need. I think having a
repository where ivy files can be shared is great - it's just icing that we
can have a plugin for ivy to do a lot of the repacking work.
I've noticed that you put up an initial list of ivy files. Do you have a
process for adding more? Submit patches to the issue tracker?
Also, a general request - does anyone have a hibernate ivy file they would
like to contribute? That's always been the one I started with and have
always stopped with it due to lack of time to figure everything out.
Chris....
Archie Cobbs wrote:
> Hello fellow Ivy users,
>
> I'd like to announce a new little project I've started, and ask for your
> feedback (and help, if interested).
>
> This project has two basic parts...
>
> 1. *Builder Resolver*<http://ivyroundup.googlecode.com/svn/wiki/files/builder.html>:
> a new Ivy resolver that accesses ivy files and "build instructions" from an
> online "builder" repository. "Builder" repositories contain ivy.xml files
> but no artifacts. To get the artifacts, the build instructions are
> downloaded from the repository and executed locally. These instructions
> specify additional resource(s) to download and how to build the artifacts
> from them, for example, by downloading a project's original distribution
> archive directly from their web site and extracting the desired artifacts.
> 2. *Ivy RoundUp Repository* <http://ivyroundup.googlecode.com/>: an
> online, open-source community "Builder" repository for all Ivy users.
>
> Please click the links for more info and documentation.
>
> I am lobbying to get the builder resolver added into Ivy itself; right now
> it's still in patch form (you can download a pre-built ivy.jar from the
> project website).
>
> Some motivations for starting this project:
>
> 1. Ivyrep is no longer maintained, but we need a decent community Ivy
> repository that everyone can share
> 2. Hosting hundreds of large files that are just copies of the same
> files available elsewhere is expensive and redundant, so let's avoid doing
> that
> 3. 99% of projects out there do not publish ivy.xml files, so we need
> a community project that focuses on developing and maintaining them
> 4. To get the most out of Ivy, there needs to be a consistent set of
> guidelines for creating ivy.xml files: how to choose organization names,
> philosophy for defining configurations, etc. A community project supported
> by Ivy users can provide this.
>
> What I want to do is gauge interest in this idea and ask for any volunteers
> who'd like to start adding and maintaining meta-data for their favorite
> projects. The Ivy RoundUp repository is online now, though only as a
> proof-of-concept (it only contains a few modules so far). Take a look and
> you should be able to get the general idea:
> http://ivyroundup.googlecode.com/
>
> In the worst case, if nobody else is interested, I will just use this for
> myself -- it's already working better than what I was doing (i.e., checking
> in giant ZIP files into Subversion and creating a project for every one to
> publish into our private Ivy repository), and in any case the work of
> setting it up is already done. Note also anyone could create their own
> private builder repository using this project as well.
>
> In the best case, we'll put together a piece of infrastructure that all Ivy
> users can really benefit from.
>
> Let me know what you think.
>
> Thanks,
> -Archie
>