You are viewing a plain text version of this content. The canonical link for it is here.
Posted to ivy-user@ant.apache.org by Andrew Thorburn <nz...@gmail.com> on 2010/04/08 11:46:29 UTC
ivy:install task - Comments/Criticisms/Bugs
I've been spending a fair amount of time with the Ivy install task
recently, attempting to set up a repository for work. I had one
previously, but I had managed that by hand. That worked fine, except
that the ivy files I produced were incompatible with the ones produced
by the ivy:install task...
Part of the reason for using the ivy:install task in the first place
is that I needed to include Axis 2 + Rampart in a project, and I sure
as hell didn't want to manually process all those JARs, so I've spent
a while trying to get this working, both for Axis, and for the other
JARs the project uses. As such, I've made the following observations:
1) The Maven 2 repository has more than a little crap in it. I don't
mean this in the sense of inconsistent organisation/module names
(which can generally be fixed with namespaces), but more that there
are several projects out there that make use of other repositories,
and I'm not entirely sure why (opensaml is the main offender here, in
my eyes, though Jasper Reports has also done it), which just makes it
a pain to copy a project with lots of dependencies, like Jasper
(funny, that). I'm not sure what Ivy can do about that, except,
perhaps, automatically adding repositories defined in a POM as a
source for all non-transitive dependencies (except those dependencies
which are actually *on* that repository, of course).
The install task is really limiting. Really, really limiting. Here's a
quick list of things I would like to be able to do:
2) Ability to specify multiple modules at once, and to say 'don't
overwrite and don't fail if present', which achieves a few things.
First, I'm installing more than one module here (roughly 20), and I'm
not super-happy about the fact that I have to specify one install task
for each, as that typically results in multiple versions of projects
being installed (Apaches commons is especially prone to that). Now,
this might someones be a good thing, but I know that all the modules
I'm installing are going to be used by a single project, and it's just
annoying to have more than one installed. The other problem is that
each install task will install another copy of a library that already
exists with the same version (e.g. commons-logging 1.1.1), and while
that's not a problem while developing the repository, I'm not sure I'd
like that to happen when installing to the company repository. It also
means unnecessary downloads of artifacts.
3) Ability to exclude/ignore dependencies. Log4J, for example, depends
on various Sun libraries that aren't included on any repository, so to
get it to install without error, I have to set transitive="false" and
add the other dependencies as yet more install tasks. There are also
other cases where I might not want to try and install a dependency.
I'm not sure if I would want them to be excluded from the Ivy file, or
simply not downloaded, but still included in the Ivy file.
4) Ability to manually specify publications. Case in point:
net.sf.json-lib#json-lib;2.3 - that does not have the standard
artifacts (1 jar, 1 source, 1 javadoc), rather it has two 'binary'
(JARs with class files) artifacts (json-lib-2.3-jdk15.jar, and
json-lib-2.3-jdk13.jar), each with their own sources and javadocs. At
this stage, I'm not sure how I'd go about installing them with the
install task. In fact, I'm quite sure I can't, and that I will have to
install it manually. Bugger.
5) Ability to revise revisions. At present, if I want all projects to
have at least, say, a 3 digit revision number (e.g. 1.2.3), that's not
possible as I get the following error when the following rule is
applied:
public: bad revision found in
http://repo1.maven.org/maven2/junit/junit/4.7/junit-4.7.pom:
expected='4.7 found='4.7.0'
<rule>
<fromsystem>
<src org="org.junit" module="junit" rev="4.7.0" />
<dest org="$m0" module="$m0" rev="4.7" />
</fromsystem>
<tosystem>
<src org="junit" module="junit" rev="4.7" />
<dest org="org.$o0" module="$m0" rev="4.7.0" />
</tosystem>
</rule>
I'm fairly sure that's a bug, but I'm not certain.
6) Not having to specify parent projects first. e.g. I have to do the following:
<ivy:install organisation="org.apache.ws" module="xmlrpc"
revision="3.0" from="installerChain" to="local" transitive="true"
overwrite="true" />
Before
<ivy:install organisation="org.apache.ws" module="xmlrpc-common"
revision="3.0" from="installerChain" to="local" transitive="true"
overwrite="true" />
Otherwise it fails with an error similar to 5, provided I'm using a
rule to change something about the project. In this case, changing the
organisation from org.apache.xmlrpc to org.apache.ws. Again, fairly
sure that's a bug, but not certain.
7) This is almost certainly an Ant problem, but I mention it here
because it drove me up the wall: My work proxy (squid) caches things,
like Ivy files. I made a change to an ivy.xml file for a particular
project (before I decided to Nuke the whole repository and start
again), and it wouldn't pick it up. Took me ages to work out that it
was being cached by squid. In the end, I didn't see a way to make ant
send the correct headers, so I had to do a 'wget --no-cache
http://path/to/file.xml' to get Ivy to pick it up... Very, very
frustrating... Need a way to tell Ant that it should include the
No-Caching headers or something...
8) If chainrules is false, then the first matching rule that applies
should be the only one used. This matters in the following case:
<rule>
<fromsystem>
<src org="org.apache.commons" module="commons-.+" />
<dest org="$m0" module="$m0" />
</fromsystem>
<tosystem>
<src org="commons-.+" module="commons-.+" />
<dest org="org.apache.commons" module="$m0" />
</tosystem>
</rule>
Now, that rule is fairly innocuous, and would work as expected if
there were not commons-parent projects in the org.apache.commons
organisation in the Maven Repo (see #1 about the repo being crap). So
this rule applies the fromsystem rule for the parent projects,
resulting in commons-parent#commons-parent;7 - oops.
I figured I could fix that by using the following rule:
<rule>
<fromsystem>
<src org="org.apache.commons" module="commons-parent" />
<dest org="$o0" module="$m0" />
</fromsystem>
<tosystem>
<src org="org.apache.commons" module="commons-parent" />
<dest org="$o0" module="$m0" />
</tosystem>
</rule>
But that doesn't work - it seems to apply both rules, regardless of
which way around they are (I think), which means I'm forced to use
this:
<rule>
<fromsystem>
<src org="org.apache.commons" module="(commons-attributes).*" />
<src org="org.apache.commons" module="(commons-beanutils).*" />
<src org="org.apache.commons" module="(commons-betwixt)" />
<src org="org.apache.commons" module="(commons-chain)" />
<src org="org.apache.commons" module="(commons-cli)" />
<src org="org.apache.commons" module="(commons-codec)" />
<src org="org.apache.commons" module="(commons-collections)" />
<src org="org.apache.commons" module="(commons-compress)" />
<src org="org.apache.commons" module="(commons-configuration)" />
<src org="org.apache.commons" module="(commons-daemon)" />
<src org="org.apache.commons" module="(commons-dbcp)" />
<src org="org.apache.commons" module="(commons-dbutils)" />
<src org="org.apache.commons" module="(commons-digester)" />
<src org="org.apache.commons" module="(commons-discovery)" />
<src org="org.apache.commons" module="(commons-el)" />
<src org="org.apache.commons" module="(commons-email)" />
<src org="org.apache.commons" module="(commons-exec)" />
<src org="org.apache.commons" module="(commons-fileupload)" />
<src org="org.apache.commons" module="(commons-httpclient)" />
<src org="org.apache.commons" module="(commons-io)" />
<src org="org.apache.commons" module="(commons-jci)" />
<src org="org.apache.commons" module="(commons-jelly)" />
<src org="org.apache.commons" module="(commons-jexl)" />
<src org="org.apache.commons" module="(commons-jxpath)" />
<src org="org.apache.commons" module="(commons-lang)" />
<src org="org.apache.commons" module="(commons-launcher)" />
<src org="org.apache.commons" module="(commons-logging)" />
<src org="org.apache.commons" module="(commons-math)" />
<src org="org.apache.commons" module="(commons-modeler)" />
<src org="org.apache.commons" module="(commons-net)" />
<src org="org.apache.commons" module="(commons-pool)" />
<src org="org.apache.commons" module="(commons-primitives)" />
<src org="org.apache.commons" module="(commons-proxy)" />
<src org="org.apache.commons" module="(commons-sanselan)" />
<src org="org.apache.commons" module="(commons-scxml)" />
<src org="org.apache.commons" module="(commons-transaction)" />
<src org="org.apache.commons" module="(commons-validator)" />
<src org="org.apache.commons" module="(commons-vfs)" />
<dest org="$m1" module="$m0" />
</fromsystem>
<tosystem>
<src org="commons-.+" module="commons-.+" />
<dest org="org.apache.commons" module="$m0" />
</tosystem>
</rule>
Which is a little ugly.
9) Support relocations or the ability to override versions. e.g.
xml-apis#xml-apis;2.0.2 has been relocated to
xml-apis#xml-apis;1.0.b2, but Ivy fails, telling me to depend directly
on the new version. Well, if I could, that would be awesome, but I
can't because I have no choice about what revision a transitive
dependency uses. So, I have to use the following rule instead:
<rule>
<fromsystem>
<src org="org.apache.xml" module="xml-apis"/>
<dest org="$m0" />
</fromsystem>
<tosystem>
<src org="xml-apis" module="xml-apis" rev="2.0.2" />
<dest org="org.apache.xml" module="$m0" rev="1.0.b2" />
</tosystem>
</rule>
Which works, despite #6 above not working.
10) Support installing source and javadoc artifacts transitively. I
know that it installs them for the artifact I've declared in the
install task, which is nice, but I really, really wish it would
install them for all artifacts. As it is, it's now a 3-step process
instead of a one-step process (1: Ivy install task of top-level
dependency to get all transitive dependencies, 2: run script to get
org/module/revision for all installed modules and generate an ant file
just for them, 3: run the new ant file to install all dependencies,
this time with source/javadoc artifacts but transitive set to false).
So creating an ivy repository from the maven repository works, but
it's not a simple process. There are things that I will have to
install manually (some of which aren't on any public repository, some
of Ivy just doesn't like), and there are way too many repositories
around for my liking - I have to have 6 to get everything I want
installed. And installing source/javadoc artifacts is a pain.
I still like Ivy, and I think it's more flexible than Maven, but it
seems like it still has a long way to go before setting up a
repository is actually a simple operation.
If you would like to raise a Jira ticket for anything here, let me
know. I think some of them may already have Jira tickets, but I'm not
certain about that.
Thanks,
- Andrew Thorburn
Re: ivy:install task - Comments/Criticisms/Bugs
Posted by Andrew Thorburn <nz...@gmail.com>.
Ok, I've created issues 1182 - 1187, for issues that haven't been
reported yet and that could be reasonably solved by Ivy.
- Andrew
On Wed, Apr 21, 2010 at 9:37 AM, Maarten Coene <ma...@yahoo.com> wrote:
> Wow, thanks for this detailed feedback.
>
> I think the best thing you can do is to create individual JIRA issues for the points you think that could get addressed by Ivy.
> https://issues.apache.org/jira/browse/IVY
>
> Maarten
>
>
>
> ----- Original Message ----
> From: Andrew Thorburn <nz...@gmail.com>
> To: ivy-user@ant.apache.org
> Sent: Thu, April 8, 2010 11:46:29 AM
> Subject: ivy:install task - Comments/Criticisms/Bugs
>
> I've been spending a fair amount of time with the Ivy install task
> recently, attempting to set up a repository for work. I had one
> previously, but I had managed that by hand. That worked fine, except
> that the ivy files I produced were incompatible with the ones produced
> by the ivy:install task...
>
> Part of the reason for using the ivy:install task in the first place
> is that I needed to include Axis 2 + Rampart in a project, and I sure
> as hell didn't want to manually process all those JARs, so I've spent
> a while trying to get this working, both for Axis, and for the other
> JARs the project uses. As such, I've made the following observations:
>
> 1) The Maven 2 repository has more than a little crap in it. I don't
> mean this in the sense of inconsistent organisation/module names
> (which can generally be fixed with namespaces), but more that there
> are several projects out there that make use of other repositories,
> and I'm not entirely sure why (opensaml is the main offender here, in
> my eyes, though Jasper Reports has also done it), which just makes it
> a pain to copy a project with lots of dependencies, like Jasper
> (funny, that). I'm not sure what Ivy can do about that, except,
> perhaps, automatically adding repositories defined in a POM as a
> source for all non-transitive dependencies (except those dependencies
> which are actually *on* that repository, of course).
>
> The install task is really limiting. Really, really limiting. Here's a
> quick list of things I would like to be able to do:
>
> 2) Ability to specify multiple modules at once, and to say 'don't
> overwrite and don't fail if present', which achieves a few things.
> First, I'm installing more than one module here (roughly 20), and I'm
> not super-happy about the fact that I have to specify one install task
> for each, as that typically results in multiple versions of projects
> being installed (Apaches commons is especially prone to that). Now,
> this might someones be a good thing, but I know that all the modules
> I'm installing are going to be used by a single project, and it's just
> annoying to have more than one installed. The other problem is that
> each install task will install another copy of a library that already
> exists with the same version (e.g. commons-logging 1.1.1), and while
> that's not a problem while developing the repository, I'm not sure I'd
> like that to happen when installing to the company repository. It also
> means unnecessary downloads of artifacts.
>
> 3) Ability to exclude/ignore dependencies. Log4J, for example, depends
> on various Sun libraries that aren't included on any repository, so to
> get it to install without error, I have to set transitive="false" and
> add the other dependencies as yet more install tasks. There are also
> other cases where I might not want to try and install a dependency.
> I'm not sure if I would want them to be excluded from the Ivy file, or
> simply not downloaded, but still included in the Ivy file.
>
> 4) Ability to manually specify publications. Case in point:
> net.sf.json-lib#json-lib;2.3 - that does not have the standard
> artifacts (1 jar, 1 source, 1 javadoc), rather it has two 'binary'
> (JARs with class files) artifacts (json-lib-2.3-jdk15.jar, and
> json-lib-2.3-jdk13.jar), each with their own sources and javadocs. At
> this stage, I'm not sure how I'd go about installing them with the
> install task. In fact, I'm quite sure I can't, and that I will have to
> install it manually. Bugger.
>
> 5) Ability to revise revisions. At present, if I want all projects to
> have at least, say, a 3 digit revision number (e.g. 1.2.3), that's not
> possible as I get the following error when the following rule is
> applied:
>
> public: bad revision found in
> http://repo1.maven.org/maven2/junit/junit/4.7/junit-4.7.pom:
> expected='4.7 found='4.7.0'
>
> <rule>
> <fromsystem>
> <src org="org.junit" module="junit" rev="4.7.0" />
> <dest org="$m0" module="$m0" rev="4.7" />
> </fromsystem>
> <tosystem>
> <src org="junit" module="junit" rev="4.7" />
> <dest org="org.$o0" module="$m0" rev="4.7.0" />
> </tosystem>
> </rule>
>
>
> I'm fairly sure that's a bug, but I'm not certain.
>
> 6) Not having to specify parent projects first. e.g. I have to do the following:
>
> <ivy:install organisation="org.apache.ws" module="xmlrpc"
> revision="3.0" from="installerChain" to="local" transitive="true"
> overwrite="true" />
>
> Before
>
> <ivy:install organisation="org.apache.ws" module="xmlrpc-common"
> revision="3.0" from="installerChain" to="local" transitive="true"
> overwrite="true" />
>
> Otherwise it fails with an error similar to 5, provided I'm using a
> rule to change something about the project. In this case, changing the
> organisation from org.apache.xmlrpc to org.apache.ws. Again, fairly
> sure that's a bug, but not certain.
>
> 7) This is almost certainly an Ant problem, but I mention it here
> because it drove me up the wall: My work proxy (squid) caches things,
> like Ivy files. I made a change to an ivy.xml file for a particular
> project (before I decided to Nuke the whole repository and start
> again), and it wouldn't pick it up. Took me ages to work out that it
> was being cached by squid. In the end, I didn't see a way to make ant
> send the correct headers, so I had to do a 'wget --no-cache
> http://path/to/file.xml' to get Ivy to pick it up... Very, very
> frustrating... Need a way to tell Ant that it should include the
> No-Caching headers or something...
>
> 8) If chainrules is false, then the first matching rule that applies
> should be the only one used. This matters in the following case:
>
> <rule>
> <fromsystem>
> <src org="org.apache.commons" module="commons-.+" />
> <dest org="$m0" module="$m0" />
> </fromsystem>
> <tosystem>
> <src org="commons-.+" module="commons-.+" />
> <dest org="org.apache.commons" module="$m0" />
> </tosystem>
> </rule>
>
> Now, that rule is fairly innocuous, and would work as expected if
> there were not commons-parent projects in the org.apache.commons
> organisation in the Maven Repo (see #1 about the repo being crap). So
> this rule applies the fromsystem rule for the parent projects,
> resulting in commons-parent#commons-parent;7 - oops.
>
> I figured I could fix that by using the following rule:
>
> <rule>
> <fromsystem>
> <src org="org.apache.commons" module="commons-parent" />
> <dest org="$o0" module="$m0" />
> </fromsystem>
> <tosystem>
> <src org="org.apache.commons" module="commons-parent" />
> <dest org="$o0" module="$m0" />
> </tosystem>
> </rule>
>
> But that doesn't work - it seems to apply both rules, regardless of
> which way around they are (I think), which means I'm forced to use
> this:
>
> <rule>
> <fromsystem>
> <src org="org.apache.commons" module="(commons-attributes).*" />
> <src org="org.apache.commons" module="(commons-beanutils).*" />
> <src org="org.apache.commons" module="(commons-betwixt)" />
> <src org="org.apache.commons" module="(commons-chain)" />
> <src org="org.apache.commons" module="(commons-cli)" />
> <src org="org.apache.commons" module="(commons-codec)" />
> <src org="org.apache.commons" module="(commons-collections)" />
> <src org="org.apache.commons" module="(commons-compress)" />
> <src org="org.apache.commons" module="(commons-configuration)" />
> <src org="org.apache.commons" module="(commons-daemon)" />
> <src org="org.apache.commons" module="(commons-dbcp)" />
> <src org="org.apache.commons" module="(commons-dbutils)" />
> <src org="org.apache.commons" module="(commons-digester)" />
> <src org="org.apache.commons" module="(commons-discovery)" />
> <src org="org.apache.commons" module="(commons-el)" />
> <src org="org.apache.commons" module="(commons-email)" />
> <src org="org.apache.commons" module="(commons-exec)" />
> <src org="org.apache.commons" module="(commons-fileupload)" />
> <src org="org.apache.commons" module="(commons-httpclient)" />
> <src org="org.apache.commons" module="(commons-io)" />
> <src org="org.apache.commons" module="(commons-jci)" />
> <src org="org.apache.commons" module="(commons-jelly)" />
> <src org="org.apache.commons" module="(commons-jexl)" />
> <src org="org.apache.commons" module="(commons-jxpath)" />
> <src org="org.apache.commons" module="(commons-lang)" />
> <src org="org.apache.commons" module="(commons-launcher)" />
> <src org="org.apache.commons" module="(commons-logging)" />
> <src org="org.apache.commons" module="(commons-math)" />
> <src org="org.apache.commons" module="(commons-modeler)" />
> <src org="org.apache.commons" module="(commons-net)" />
> <src org="org.apache.commons" module="(commons-pool)" />
> <src org="org.apache.commons" module="(commons-primitives)" />
> <src org="org.apache.commons" module="(commons-proxy)" />
> <src org="org.apache.commons" module="(commons-sanselan)" />
> <src org="org.apache.commons" module="(commons-scxml)" />
> <src org="org.apache.commons" module="(commons-transaction)" />
> <src org="org.apache.commons" module="(commons-validator)" />
> <src org="org.apache.commons" module="(commons-vfs)" />
> <dest org="$m1" module="$m0" />
> </fromsystem>
> <tosystem>
> <src org="commons-.+" module="commons-.+" />
> <dest org="org.apache.commons" module="$m0" />
> </tosystem>
> </rule>
>
> Which is a little ugly.
>
> 9) Support relocations or the ability to override versions. e.g.
> xml-apis#xml-apis;2.0.2 has been relocated to
> xml-apis#xml-apis;1.0.b2, but Ivy fails, telling me to depend directly
> on the new version. Well, if I could, that would be awesome, but I
> can't because I have no choice about what revision a transitive
> dependency uses. So, I have to use the following rule instead:
>
> <rule>
> <fromsystem>
> <src org="org.apache.xml" module="xml-apis"/>
> <dest org="$m0" />
> </fromsystem>
> <tosystem>
> <src org="xml-apis" module="xml-apis" rev="2.0.2" />
> <dest org="org.apache.xml" module="$m0" rev="1.0.b2" />
> </tosystem>
> </rule>
>
> Which works, despite #6 above not working.
>
> 10) Support installing source and javadoc artifacts transitively. I
> know that it installs them for the artifact I've declared in the
> install task, which is nice, but I really, really wish it would
> install them for all artifacts. As it is, it's now a 3-step process
> instead of a one-step process (1: Ivy install task of top-level
> dependency to get all transitive dependencies, 2: run script to get
> org/module/revision for all installed modules and generate an ant file
> just for them, 3: run the new ant file to install all dependencies,
> this time with source/javadoc artifacts but transitive set to false).
>
>
> So creating an ivy repository from the maven repository works, but
> it's not a simple process. There are things that I will have to
> install manually (some of which aren't on any public repository, some
> of Ivy just doesn't like), and there are way too many repositories
> around for my liking - I have to have 6 to get everything I want
> installed. And installing source/javadoc artifacts is a pain.
>
> I still like Ivy, and I think it's more flexible than Maven, but it
> seems like it still has a long way to go before setting up a
> repository is actually a simple operation.
>
> If you would like to raise a Jira ticket for anything here, let me
> know. I think some of them may already have Jira tickets, but I'm not
> certain about that.
>
> Thanks,
>
> - Andrew Thorburn
>
>
>
>
>
Re: ivy:install task - Comments/Criticisms/Bugs
Posted by Maarten Coene <ma...@yahoo.com>.
Wow, thanks for this detailed feedback.
I think the best thing you can do is to create individual JIRA issues for the points you think that could get addressed by Ivy.
https://issues.apache.org/jira/browse/IVY
Maarten
----- Original Message ----
From: Andrew Thorburn <nz...@gmail.com>
To: ivy-user@ant.apache.org
Sent: Thu, April 8, 2010 11:46:29 AM
Subject: ivy:install task - Comments/Criticisms/Bugs
I've been spending a fair amount of time with the Ivy install task
recently, attempting to set up a repository for work. I had one
previously, but I had managed that by hand. That worked fine, except
that the ivy files I produced were incompatible with the ones produced
by the ivy:install task...
Part of the reason for using the ivy:install task in the first place
is that I needed to include Axis 2 + Rampart in a project, and I sure
as hell didn't want to manually process all those JARs, so I've spent
a while trying to get this working, both for Axis, and for the other
JARs the project uses. As such, I've made the following observations:
1) The Maven 2 repository has more than a little crap in it. I don't
mean this in the sense of inconsistent organisation/module names
(which can generally be fixed with namespaces), but more that there
are several projects out there that make use of other repositories,
and I'm not entirely sure why (opensaml is the main offender here, in
my eyes, though Jasper Reports has also done it), which just makes it
a pain to copy a project with lots of dependencies, like Jasper
(funny, that). I'm not sure what Ivy can do about that, except,
perhaps, automatically adding repositories defined in a POM as a
source for all non-transitive dependencies (except those dependencies
which are actually *on* that repository, of course).
The install task is really limiting. Really, really limiting. Here's a
quick list of things I would like to be able to do:
2) Ability to specify multiple modules at once, and to say 'don't
overwrite and don't fail if present', which achieves a few things.
First, I'm installing more than one module here (roughly 20), and I'm
not super-happy about the fact that I have to specify one install task
for each, as that typically results in multiple versions of projects
being installed (Apaches commons is especially prone to that). Now,
this might someones be a good thing, but I know that all the modules
I'm installing are going to be used by a single project, and it's just
annoying to have more than one installed. The other problem is that
each install task will install another copy of a library that already
exists with the same version (e.g. commons-logging 1.1.1), and while
that's not a problem while developing the repository, I'm not sure I'd
like that to happen when installing to the company repository. It also
means unnecessary downloads of artifacts.
3) Ability to exclude/ignore dependencies. Log4J, for example, depends
on various Sun libraries that aren't included on any repository, so to
get it to install without error, I have to set transitive="false" and
add the other dependencies as yet more install tasks. There are also
other cases where I might not want to try and install a dependency.
I'm not sure if I would want them to be excluded from the Ivy file, or
simply not downloaded, but still included in the Ivy file.
4) Ability to manually specify publications. Case in point:
net.sf.json-lib#json-lib;2.3 - that does not have the standard
artifacts (1 jar, 1 source, 1 javadoc), rather it has two 'binary'
(JARs with class files) artifacts (json-lib-2.3-jdk15.jar, and
json-lib-2.3-jdk13.jar), each with their own sources and javadocs. At
this stage, I'm not sure how I'd go about installing them with the
install task. In fact, I'm quite sure I can't, and that I will have to
install it manually. Bugger.
5) Ability to revise revisions. At present, if I want all projects to
have at least, say, a 3 digit revision number (e.g. 1.2.3), that's not
possible as I get the following error when the following rule is
applied:
public: bad revision found in
http://repo1.maven.org/maven2/junit/junit/4.7/junit-4.7.pom:
expected='4.7 found='4.7.0'
<rule>
<fromsystem>
<src org="org.junit" module="junit" rev="4.7.0" />
<dest org="$m0" module="$m0" rev="4.7" />
</fromsystem>
<tosystem>
<src org="junit" module="junit" rev="4.7" />
<dest org="org.$o0" module="$m0" rev="4.7.0" />
</tosystem>
</rule>
I'm fairly sure that's a bug, but I'm not certain.
6) Not having to specify parent projects first. e.g. I have to do the following:
<ivy:install organisation="org.apache.ws" module="xmlrpc"
revision="3.0" from="installerChain" to="local" transitive="true"
overwrite="true" />
Before
<ivy:install organisation="org.apache.ws" module="xmlrpc-common"
revision="3.0" from="installerChain" to="local" transitive="true"
overwrite="true" />
Otherwise it fails with an error similar to 5, provided I'm using a
rule to change something about the project. In this case, changing the
organisation from org.apache.xmlrpc to org.apache.ws. Again, fairly
sure that's a bug, but not certain.
7) This is almost certainly an Ant problem, but I mention it here
because it drove me up the wall: My work proxy (squid) caches things,
like Ivy files. I made a change to an ivy.xml file for a particular
project (before I decided to Nuke the whole repository and start
again), and it wouldn't pick it up. Took me ages to work out that it
was being cached by squid. In the end, I didn't see a way to make ant
send the correct headers, so I had to do a 'wget --no-cache
http://path/to/file.xml' to get Ivy to pick it up... Very, very
frustrating... Need a way to tell Ant that it should include the
No-Caching headers or something...
8) If chainrules is false, then the first matching rule that applies
should be the only one used. This matters in the following case:
<rule>
<fromsystem>
<src org="org.apache.commons" module="commons-.+" />
<dest org="$m0" module="$m0" />
</fromsystem>
<tosystem>
<src org="commons-.+" module="commons-.+" />
<dest org="org.apache.commons" module="$m0" />
</tosystem>
</rule>
Now, that rule is fairly innocuous, and would work as expected if
there were not commons-parent projects in the org.apache.commons
organisation in the Maven Repo (see #1 about the repo being crap). So
this rule applies the fromsystem rule for the parent projects,
resulting in commons-parent#commons-parent;7 - oops.
I figured I could fix that by using the following rule:
<rule>
<fromsystem>
<src org="org.apache.commons" module="commons-parent" />
<dest org="$o0" module="$m0" />
</fromsystem>
<tosystem>
<src org="org.apache.commons" module="commons-parent" />
<dest org="$o0" module="$m0" />
</tosystem>
</rule>
But that doesn't work - it seems to apply both rules, regardless of
which way around they are (I think), which means I'm forced to use
this:
<rule>
<fromsystem>
<src org="org.apache.commons" module="(commons-attributes).*" />
<src org="org.apache.commons" module="(commons-beanutils).*" />
<src org="org.apache.commons" module="(commons-betwixt)" />
<src org="org.apache.commons" module="(commons-chain)" />
<src org="org.apache.commons" module="(commons-cli)" />
<src org="org.apache.commons" module="(commons-codec)" />
<src org="org.apache.commons" module="(commons-collections)" />
<src org="org.apache.commons" module="(commons-compress)" />
<src org="org.apache.commons" module="(commons-configuration)" />
<src org="org.apache.commons" module="(commons-daemon)" />
<src org="org.apache.commons" module="(commons-dbcp)" />
<src org="org.apache.commons" module="(commons-dbutils)" />
<src org="org.apache.commons" module="(commons-digester)" />
<src org="org.apache.commons" module="(commons-discovery)" />
<src org="org.apache.commons" module="(commons-el)" />
<src org="org.apache.commons" module="(commons-email)" />
<src org="org.apache.commons" module="(commons-exec)" />
<src org="org.apache.commons" module="(commons-fileupload)" />
<src org="org.apache.commons" module="(commons-httpclient)" />
<src org="org.apache.commons" module="(commons-io)" />
<src org="org.apache.commons" module="(commons-jci)" />
<src org="org.apache.commons" module="(commons-jelly)" />
<src org="org.apache.commons" module="(commons-jexl)" />
<src org="org.apache.commons" module="(commons-jxpath)" />
<src org="org.apache.commons" module="(commons-lang)" />
<src org="org.apache.commons" module="(commons-launcher)" />
<src org="org.apache.commons" module="(commons-logging)" />
<src org="org.apache.commons" module="(commons-math)" />
<src org="org.apache.commons" module="(commons-modeler)" />
<src org="org.apache.commons" module="(commons-net)" />
<src org="org.apache.commons" module="(commons-pool)" />
<src org="org.apache.commons" module="(commons-primitives)" />
<src org="org.apache.commons" module="(commons-proxy)" />
<src org="org.apache.commons" module="(commons-sanselan)" />
<src org="org.apache.commons" module="(commons-scxml)" />
<src org="org.apache.commons" module="(commons-transaction)" />
<src org="org.apache.commons" module="(commons-validator)" />
<src org="org.apache.commons" module="(commons-vfs)" />
<dest org="$m1" module="$m0" />
</fromsystem>
<tosystem>
<src org="commons-.+" module="commons-.+" />
<dest org="org.apache.commons" module="$m0" />
</tosystem>
</rule>
Which is a little ugly.
9) Support relocations or the ability to override versions. e.g.
xml-apis#xml-apis;2.0.2 has been relocated to
xml-apis#xml-apis;1.0.b2, but Ivy fails, telling me to depend directly
on the new version. Well, if I could, that would be awesome, but I
can't because I have no choice about what revision a transitive
dependency uses. So, I have to use the following rule instead:
<rule>
<fromsystem>
<src org="org.apache.xml" module="xml-apis"/>
<dest org="$m0" />
</fromsystem>
<tosystem>
<src org="xml-apis" module="xml-apis" rev="2.0.2" />
<dest org="org.apache.xml" module="$m0" rev="1.0.b2" />
</tosystem>
</rule>
Which works, despite #6 above not working.
10) Support installing source and javadoc artifacts transitively. I
know that it installs them for the artifact I've declared in the
install task, which is nice, but I really, really wish it would
install them for all artifacts. As it is, it's now a 3-step process
instead of a one-step process (1: Ivy install task of top-level
dependency to get all transitive dependencies, 2: run script to get
org/module/revision for all installed modules and generate an ant file
just for them, 3: run the new ant file to install all dependencies,
this time with source/javadoc artifacts but transitive set to false).
So creating an ivy repository from the maven repository works, but
it's not a simple process. There are things that I will have to
install manually (some of which aren't on any public repository, some
of Ivy just doesn't like), and there are way too many repositories
around for my liking - I have to have 6 to get everything I want
installed. And installing source/javadoc artifacts is a pain.
I still like Ivy, and I think it's more flexible than Maven, but it
seems like it still has a long way to go before setting up a
repository is actually a simple operation.
If you would like to raise a Jira ticket for anything here, let me
know. I think some of them may already have Jira tickets, but I'm not
certain about that.
Thanks,
- Andrew Thorburn
Re: ivy:install task - Comments/Criticisms/Bugs
Posted by Archie Cobbs <ar...@gmail.com>.
Yep, OSGi is a great system. There really needs to be an effort to create a
unified repository and dependency system that combines the best aspects of
maven, ivy, and OSGi. Right now you can't get a complete solution in one
place.
-Archie
On Thu, Apr 8, 2010 at 9:50 PM, Mitch Gitman <mg...@gmail.com> wrote:
> As long as we're talking about public Ivy repositories, this is a good
> opportunity to give a shout-out to the SpringSource Enterprise Bundle
> Repository:
> http://www.springsource.com/repository/app/
>
> Do note that the various non-Spring artifacts published on Spring's Ivy
> repo
> are not literally the original artifacts but rather OSGi-ified bundles of
> them. Same JARs+extra metadata.
>
> I will say, Ivy RoundUp has its one distinctive approach to being a public
> Ivy repo--centered around the packager resolver and its own decisions as to
> Ivy confs--and the SpringSource bundle repo has its own distinctive
> approach. Neither approach is more valid and neither repo is a
> be-all-and-end-all, but actually it's nice to know that we're seeing at
> least the beginnings of a diversity of choices out there for public Ivy
> repos just as, well, there are for public Maven repos.
>
> On Thu, Apr 8, 2010 at 7:41 PM, Archie Cobbs <ar...@gmail.com>
> wrote:
>
> > On Thu, Apr 8, 2010 at 11:02 AM, Jeff Evans (IT) <
> > Jeffrey.Evans@morganstanley.com> wrote:
> >
> > > > 1) The Maven 2 repository has more than a little crap in it. <snip>
> > >
> >
> > I'll say one more thing about Ivy RoundUp and then shut up... :-)
> >
> > The whole point of Ivy RoundUp is to create a *sane* repository with
> > reliable and precise configuration and dependency information. Right now
> > there are 231 separate modules and 617 distinct revisions, but there is
> > always the need for more.
> >
> > If people want to be part of the solution (instead of just complaining
> :-)
> > please join us. Everyone benefits from the "network effect" of people
> > contributing ivy.xml files for the modules that they care about and use
> > every day. In that way it's a project very much in the open source
> spirit.
> >
> > I'll also note that you don't have to use Ivy RoundUp exclusively, as you
> > can chain it together with other repositories, so it's not an
> > all-or-nothing
> > commitment to use it (although of course maven can sometimes make this
> > difficult too).
> >
> > Thanks,
> > -Archie
> >
> > --
> > Archie L. Cobbs
> >
>
--
Archie L. Cobbs
Re: ivy:install task - Comments/Criticisms/Bugs
Posted by Mitch Gitman <mg...@gmail.com>.
As long as we're talking about public Ivy repositories, this is a good
opportunity to give a shout-out to the SpringSource Enterprise Bundle
Repository:
http://www.springsource.com/repository/app/
Do note that the various non-Spring artifacts published on Spring's Ivy repo
are not literally the original artifacts but rather OSGi-ified bundles of
them. Same JARs+extra metadata.
I will say, Ivy RoundUp has its one distinctive approach to being a public
Ivy repo--centered around the packager resolver and its own decisions as to
Ivy confs--and the SpringSource bundle repo has its own distinctive
approach. Neither approach is more valid and neither repo is a
be-all-and-end-all, but actually it's nice to know that we're seeing at
least the beginnings of a diversity of choices out there for public Ivy
repos just as, well, there are for public Maven repos.
On Thu, Apr 8, 2010 at 7:41 PM, Archie Cobbs <ar...@gmail.com> wrote:
> On Thu, Apr 8, 2010 at 11:02 AM, Jeff Evans (IT) <
> Jeffrey.Evans@morganstanley.com> wrote:
>
> > > 1) The Maven 2 repository has more than a little crap in it. <snip>
> >
>
> I'll say one more thing about Ivy RoundUp and then shut up... :-)
>
> The whole point of Ivy RoundUp is to create a *sane* repository with
> reliable and precise configuration and dependency information. Right now
> there are 231 separate modules and 617 distinct revisions, but there is
> always the need for more.
>
> If people want to be part of the solution (instead of just complaining :-)
> please join us. Everyone benefits from the "network effect" of people
> contributing ivy.xml files for the modules that they care about and use
> every day. In that way it's a project very much in the open source spirit.
>
> I'll also note that you don't have to use Ivy RoundUp exclusively, as you
> can chain it together with other repositories, so it's not an
> all-or-nothing
> commitment to use it (although of course maven can sometimes make this
> difficult too).
>
> Thanks,
> -Archie
>
> --
> Archie L. Cobbs
>
Re: ivy:install task - Comments/Criticisms/Bugs
Posted by Archie Cobbs <ar...@gmail.com>.
On Thu, Apr 8, 2010 at 11:02 AM, Jeff Evans (IT) <
Jeffrey.Evans@morganstanley.com> wrote:
> > 1) The Maven 2 repository has more than a little crap in it. <snip>
>
I'll say one more thing about Ivy RoundUp and then shut up... :-)
The whole point of Ivy RoundUp is to create a *sane* repository with
reliable and precise configuration and dependency information. Right now
there are 231 separate modules and 617 distinct revisions, but there is
always the need for more.
If people want to be part of the solution (instead of just complaining :-)
please join us. Everyone benefits from the "network effect" of people
contributing ivy.xml files for the modules that they care about and use
every day. In that way it's a project very much in the open source spirit.
I'll also note that you don't have to use Ivy RoundUp exclusively, as you
can chain it together with other repositories, so it's not an all-or-nothing
commitment to use it (although of course maven can sometimes make this
difficult too).
Thanks,
-Archie
--
Archie L. Cobbs
Re: ivy:install task - Comments/Criticisms/Bugs
Posted by Archie Cobbs <ar...@gmail.com>.
Would you mind filing a bug just listing the projects you'd like to see
included? We'll try to get them added.
Thanks,
-Archie
On Thu, Apr 8, 2010 at 8:44 PM, Andrew Thorburn <nz...@gmail.com> wrote:
> Just had a look at it, and it's not suitable as it doesn't contain all
> the packages I want (Axis2, in this case). Apart from that it looks
> quite nice. But given the number of packages I need, it's not
> appropriate for me at this stage.
>
> Thanks,
>
> - Andrew
>
> On Fri, Apr 9, 2010 at 4:49 AM, Archie Cobbs <ar...@gmail.com>
> wrote:
> > On Thu, Apr 8, 2010 at 11:02 AM, Jeff Evans (IT) <
> > Jeffrey.Evans@morganstanley.com> wrote:
> >
> >> Even better would be if there was a pure-Ivy way of starting out, that
> >> didn't rely on going to public Maven repositories, then adopting POMs
> >> into Ivy files, to get a custom repository with locally-fetched
> >> artifacts and generated Ivy files started. This would take care of a
> >> lot of the problems you are running into (and I have been as well).
> >> It would be nice if there was a public Ivy repository that contained
> >> everything (artifacts too, not just Ivy files). My guess is this
> >> doesn't exist due to a lack of interest/time/resources/ownership,
> >> which is certainly understandable.
> >>
> >
> > Just curious.. have you guys looked at Ivy
> > RoundUp<http://code.google.com/p/ivyroundup/>?
> > And if so, why or why not was it acceptable for your use?
> >
> > Thanks,
> > -Archie
> >
> > --
> > Archie L. Cobbs
> >
>
--
Archie L. Cobbs
Re: ivy:install task - Comments/Criticisms/Bugs
Posted by Andrew Thorburn <nz...@gmail.com>.
Just had a look at it, and it's not suitable as it doesn't contain all
the packages I want (Axis2, in this case). Apart from that it looks
quite nice. But given the number of packages I need, it's not
appropriate for me at this stage.
Thanks,
- Andrew
On Fri, Apr 9, 2010 at 4:49 AM, Archie Cobbs <ar...@gmail.com> wrote:
> On Thu, Apr 8, 2010 at 11:02 AM, Jeff Evans (IT) <
> Jeffrey.Evans@morganstanley.com> wrote:
>
>> Even better would be if there was a pure-Ivy way of starting out, that
>> didn't rely on going to public Maven repositories, then adopting POMs
>> into Ivy files, to get a custom repository with locally-fetched
>> artifacts and generated Ivy files started. This would take care of a
>> lot of the problems you are running into (and I have been as well).
>> It would be nice if there was a public Ivy repository that contained
>> everything (artifacts too, not just Ivy files). My guess is this
>> doesn't exist due to a lack of interest/time/resources/ownership,
>> which is certainly understandable.
>>
>
> Just curious.. have you guys looked at Ivy
> RoundUp<http://code.google.com/p/ivyroundup/>?
> And if so, why or why not was it acceptable for your use?
>
> Thanks,
> -Archie
>
> --
> Archie L. Cobbs
>
Re: ivy:install task - Comments/Criticisms/Bugs
Posted by Karolis Petrauskas <k....@gmail.com>.
Hello,
On Thu, Apr 8, 2010 at 7:49 PM, Archie Cobbs <ar...@gmail.com> wrote:
>
> Just curious.. have you guys looked at Ivy
> RoundUp<http://code.google.com/p/ivyroundup/>?
> And if so, why or why not was it acceptable for your use?
Thanks for the link. I think it would be great to have this
reference in http://ant.apache.org/ivy/links.html.
Karolis
Re: ivy:install task - Comments/Criticisms/Bugs
Posted by Archie Cobbs <ar...@gmail.com>.
On Thu, Apr 8, 2010 at 11:02 AM, Jeff Evans (IT) <
Jeffrey.Evans@morganstanley.com> wrote:
> Even better would be if there was a pure-Ivy way of starting out, that
> didn't rely on going to public Maven repositories, then adopting POMs
> into Ivy files, to get a custom repository with locally-fetched
> artifacts and generated Ivy files started. This would take care of a
> lot of the problems you are running into (and I have been as well).
> It would be nice if there was a public Ivy repository that contained
> everything (artifacts too, not just Ivy files). My guess is this
> doesn't exist due to a lack of interest/time/resources/ownership,
> which is certainly understandable.
>
Just curious.. have you guys looked at Ivy
RoundUp<http://code.google.com/p/ivyroundup/>?
And if so, why or why not was it acceptable for your use?
Thanks,
-Archie
--
Archie L. Cobbs
Re: ivy:install task - Comments/Criticisms/Bugs
Posted by "Jeff Evans (IT)" <Je...@morganstanley.com>.
> 1) The Maven 2 repository has more than a little crap in it. <snip>
That's really a problem with the Maven repository and not Ivy itself.
As much as I would love for the whole Maven integration to work more
smoothly, I can't really justify blame Ivy for not being able to
handle this.
> The install task is really limiting. Really, really limiting. Here's a
> quick list of things I would like to be able to do:
>
> 2) Ability to specify multiple modules at once, and to say 'don't
> overwrite and don't fail if present', which achieves a few things. <snip>
"Don't overwrite and don't fail if present" would be great, I agree.
As for the multiple versions thing... what can Ivy really do? Those
are a byproduct of sloppy POMs (which are outside the scope of Ivy).
If some module is saying, "I require revision 1.2.2 of Blah", Ivy
can't just assume it would work with revision 1.2.3 of Blah. The
problem should be fixed by having the module depend on a dynamic
revision of Blah, which is possible in Ivy, but again I don't know
about Maven, which it sounds like you're sourcing everything from.
> 3) Ability to exclude/ignore dependencies. Log4J, for example, depends
> on various Sun libraries that aren't included on any repository, so to
> get it to install without error, I have to set transitive="false" and
> add the other dependencies as yet more install tasks. There are also
> other cases where I might not want to try and install a dependency.
> I'm not sure if I would want them to be excluded from the Ivy file, or
> simply not downloaded, but still included in the Ivy file.
You can definitely exclude specific modules; I have done it.
http://www.google.com/search?&q=+site:ant.apache.org+ivy+exclude
> 4) Ability to manually specify publications. Case in point:
> net.sf.json-lib#json-lib;2.3 - that does not have the standard
> artifacts (1 jar, 1 source, 1 javadoc), rather it has two 'binary'
> (JARs with class files) artifacts (json-lib-2.3-jdk15.jar, and
> json-lib-2.3-jdk13.jar), each with their own sources and javadocs. At
> this stage, I'm not sure how I'd go about installing them with the
> install task. In fact, I'm quite sure I can't, and that I will have to
> install it manually. Bugger.
You can always use a dual resolver, with one of the resolvers pointing
to the public URL, and another local filesystem resolver to pick up
the "missing artifacts", which you would manually place there.
Still, if the POM is well-defined (again, a big "if"), I see no reason
why the project you mentioned shouldn't get resolved and installed
correctly by an ibiblio resolver. I should just point out that you
probably only want one of those jar/source pairs depending on what JDK
you are using (1.3 versus 1.5).
> 5) Ability to revise revisions. At present, if I want all projects to
> have at least, say, a 3 digit revision number (e.g. 1.2.3), that's not
> possible as I get the following error when the following rule is
> applied:
>
> public: bad revision found in
> http://repo1.maven.org/maven2/junit/junit/4.7/junit-4.7.pom:
> expected='4.7 found='4.7.0'
That error is not about Ivy being unable to support 3-part revision
numbers. It happens because the revision number it was given does not
agree with what was found in the POM. From my experience this is
almost always because of incorrect rules.
> 6) Not having to specify parent projects first. e.g. I have to do the following:
>
> <ivy:install organisation="org.apache.ws" module="xmlrpc"
> revision="3.0" from="installerChain" to="local" transitive="true"
> overwrite="true" />
>
> Before
>
> <ivy:install organisation="org.apache.ws" module="xmlrpc-common"
> revision="3.0" from="installerChain" to="local" transitive="true"
> overwrite="true" />
>
> Otherwise it fails with an error similar to 5, provided I'm using a
> rule to change something about the project. In this case, changing the
> organisation from org.apache.xmlrpc to org.apache.ws. Again, fairly
> sure that's a bug, but not certain.
This again has to do with inconsistent Maven POM data. Ivy
namespaces/rules are a convenient way to deal with these
inconsistencies, but they are definitely hard to get right.
> 8) If chainrules is false, then the first matching rule that applies
> should be the only one used. This matters in the following case: <snip>
That's the exact behavior I have seen (first applicable matching rule
is the only one used without chainrules="true"). Perhaps there is
something else going on with your settings? Maybe you should paste
them somewhere.
> 9) Support relocations or the ability to override versions. e.g.
> xml-apis#xml-apis;2.0.2 has been relocated to
> xml-apis#xml-apis;1.0.b2, but Ivy fails, telling me to depend directly
> on the new version. <snip>
That's again an unfortunate consequence of bad Maven metadata being
adapted into Ivy. You can always argue the ibiblio resolver should
handle cases like this more elegantly, but at the end of the day it's
trying to get one system working with another's metadata. There are
going to be such issues.
> I still like Ivy, and I think it's more flexible than Maven, but it
> seems like it still has a long way to go before setting up a
> repository is actually a simple operation.
I agree with the crux of what you're saying. The Ivy website provides
nice tutorials on building a custom repository, including setting up
some ibiblio resolvers and a starting point for the rules, etc. But
it doesn't go in to much depth about dealing with those "real world"
issues one encounters when taking such an approach, like the ones
you've run into (bad POMs, inconsistent or too-specific revision
numbers in POMs, dealing with the whole Maven vs Maven2 naming thing,
figuring out the set of rules required to get things working, etc.).
It would be fantastic if someone would provide such a tutorial and
kept it up-to-date for different scenarios.
Even better would be if there was a pure-Ivy way of starting out, that
didn't rely on going to public Maven repositories, then adopting POMs
into Ivy files, to get a custom repository with locally-fetched
artifacts and generated Ivy files started. This would take care of a
lot of the problems you are running into (and I have been as well).
It would be nice if there was a public Ivy repository that contained
everything (artifacts too, not just Ivy files). My guess is this
doesn't exist due to a lack of interest/time/resources/ownership,
which is certainly understandable.