You are viewing a plain text version of this content. The canonical link for it is here.
Posted to ivy-user@ant.apache.org by Burt Leung <bu...@gmail.com> on 2010/01/06 01:18:01 UTC
Redundant JAR files in configuration groups
Hi All,
This is sort of both a question and a suggestion.
The team that I am on is still in the process of (starting to) adopt Ivy as
part of our development process. One of the key concerns raised was the fact
that the use of configurations "groups" can produce redundant JAR files in
the lib folder.
For example, I currently have configurations defined for one of our modules
like so:
================
<configurations>
<conf name="compileSrcConf" description="Configuration for compiling
the source code."/>
<conf name="default" extends="compileSrcConf" description="Default
configuration"/>
<conf name="compileTestsConf" extends="compileSrcConf"
description="Configuration for compiling the test code."/>
<conf name="checkstyleConf" extends="compileSrcConf"
description="Configuration for running checkstyle."/>
</configurations>
<dependencies>
<dependency org="someDependencyA" name="collections" rev="1.0"
conf="compileSrcConf->default"/>
<dependency org="someDependencyB" name="collections" rev="1.0"
conf="compileSrcConf->default"/>
<dependency org="someDependencyC" name="collections" rev="1.0"
conf="compileSrcConf->default"/>
<dependency org="someDependencyD" name="collections" rev="1.0"
conf="compileSrcConf->default"/>
<dependency org="someDependencyE" name="collections" rev="1.0"
conf="compileSrcConf->default"/>
<dependency org="someDependencyF" name="collections" rev="1.0"
conf="compileSrcConf->default"/>
<dependency org="checkstyle" name="checkstyle" rev="4.1"
conf="checkstyleConf->default"/>
<dependency org="junit" name="junit" rev="4.1"
conf="compileTestsConf->default"/>
</dependencies>
================
Because some of the configurations extend the compileSrcConf configuration,
a typical Ivy retrieve will produce a lib folder that looks like the
following:
================
/lib/compileSrcConf/
someDependencyA.jar
someDependencyB.jar
someDependencyC.jar
someDependencyD.jar
someDependencyE.jar
someDependencyF.jar
/lib/compileTestsConf/
someDependencyA.jar
someDependencyB.jar
someDependencyC.jar
someDependencyD.jar
someDependencyE.jar
someDependencyF.jar
junit.jar
/lib/checkstyleConf/
someDependencyA.jar
someDependencyB.jar
someDependencyC.jar
someDependencyD.jar
someDependencyE.jar
someDependencyF.jar
checkstyle.jar
================
We are considering removing the "extends" attribute from each configuration
declaration so that there would not be this redundency of JAR files. We'd of
course have to setup composite paths/pathelements in the ANT build file so
that this would work. For example, an excerpt for the ANT build file:
================
<property name="compileSrcLibDir"
value="${ivyResolveLibDir}/compileSrcConf"/>
<property name="compileTestsLibDir"
value="${ivyResolveLibDir}/compileTestsConf"/>
<property name="checkstyleLibDir"
value="${ivyResolveLibDir}/checkstyleConf"/>
...
<!-- The classpath used to compile the module. -->
<path id="classpath.module.compile">
<fileset dir="${compileSrcLibDir}">
<include name="**/*.jar" />
</fileset>
</path>
<!-- The classpath used to compile the tests -->
<path id="classpath.tests.compile">
<fileset dir="${compileTestsLibDir}">
<include name="**/*.jar" />
</fileset>
<path refid="classpath.module.compile" /> ***<== workaround for
not using "extends" feature in Ivy configuration.
</path>
<!-- The classpath used to compile the tests -->
<path id="classpath.checkstyle">
<fileset dir="${compileTestsLibDir}">
<include name="**/*.jar" />
</fileset>
<path refid="classpath.module.compile" /> ***<== workaround for not
using "extends" feature in Ivy configuration.
</path>
================
Question: is there a good argument for ignoring/tolerating this redundancy
of JAR files?
I personally think it would be nice if Ivy setup/provided ANT path elements
behind the scenes based on the Ivy configuration groups. As I understand it,
when Ivy does a resolve/retrieve, it first downloads the JAR files to its
cache directory on the computer (e.g. "c:/documents and
settings/user/.ivy/"). Then it will copy the JAR files specified by the
configuration group(s) defined in a module's ivy.xml into its lib folder
using a pattern like:
pattern="${ivyResolveLibDir}/[conf]/[organization]/[artifact]-[revision].[ext]"
Since the JAR files are already in the Ivy cache directory I think that Ivy
could (automatically) create ANT pathelements behind the scenes (for each
Ivy configuration group) that could then be referenced from the ANT build
file. This would save the "copy" step I mentioned above. Does anyone else
think this would be an improvement?
Thanks,
Burt
Re: Redundant JAR files in configuration groups
Posted by Burt Leung <bu...@gmail.com>.
Hi Mitch,
Thank you very much for giving me a bit of your time and expertise!
I read the documentation for the ivy:cachepath operation and got it to work
for our project just now. As you mentioned, I no longer need to call
ivy:retrieve in our build too (which created duplicate JAR files in the /lib
folder of the module).
Cheers,
Burt
On Tue, Jan 5, 2010 at 2:44 PM, Mitch Gitman <mg...@gmail.com> wrote:
> Burt, you wrote: "I personally think it would be nice if Ivy setup/provided
> ANT path elements behind the scenes based on the Ivy configuration groups."
>
> Ivy does offer this, just not behind the scenes. Look at the ivy:cachepath
> Ant task:
> http://ant.apache.org/ivy/history/trunk/use/cachepath.html
>
> This would obviate the need to use the ivy:retrieve task and create all
> those lib directories. To me, the use of ivy:retrieve and lib directories
> signals that you've kinda, sorta gotten into Ivy, but you haven't
> completely
> bought into or embraced Ivy yet. You've created a redundant set of
> directories to make your file setup look more like the pre-Ivy days.
>
> Also, about the whole matter of confs extending confs, I know this is very
> subjective and not right-or-wrong, but to me the whole idea of having your
> test Ivy conf extend your main Ivy conf is a case of meshing two things
> together too early. What if you want to see just the dependencies that
> apply
> to test and not main? You're stuck having to sift through maybe 30 JARs to
> see just three.
>
> Instead have your test Ivy conf pick up test-only dependencies. In other
> words, don't let your test conf extend your main conf. When you need to
> combine the JARs from both confs in a path, just do it as-needed using
> composition. So using the old-fashioned ivy:retrieve and lib dirs, you'd
> have:
> <path id="classpath.tests.compile">
> <fileset dir="${compileSrcLibDir}">
> <include name="**/*.jar" />
> </fileset>
> <fileset dir="${compileTestsLibDir}">
> <include name="**/*.jar" />
> </fileset>
> </path>
>
> Or to use the more direct ivy:cachepath where ivy.src.classpath and
> ivy.test.classpath are the paths you've created therein:
> <path id="classpath.tests.compile">
> <path refid="ivy.src.classpath" />
> <path refid="ivy.test.classpath" />
> </path>
>
>
Re: Redundant JAR files in configuration groups
Posted by Mitch Gitman <mg...@gmail.com>.
Burt, you wrote: "I personally think it would be nice if Ivy setup/provided
ANT path elements behind the scenes based on the Ivy configuration groups."
Ivy does offer this, just not behind the scenes. Look at the ivy:cachepath
Ant task:
http://ant.apache.org/ivy/history/trunk/use/cachepath.html
This would obviate the need to use the ivy:retrieve task and create all
those lib directories. To me, the use of ivy:retrieve and lib directories
signals that you've kinda, sorta gotten into Ivy, but you haven't completely
bought into or embraced Ivy yet. You've created a redundant set of
directories to make your file setup look more like the pre-Ivy days.
Also, about the whole matter of confs extending confs, I know this is very
subjective and not right-or-wrong, but to me the whole idea of having your
test Ivy conf extend your main Ivy conf is a case of meshing two things
together too early. What if you want to see just the dependencies that apply
to test and not main? You're stuck having to sift through maybe 30 JARs to
see just three.
Instead have your test Ivy conf pick up test-only dependencies. In other
words, don't let your test conf extend your main conf. When you need to
combine the JARs from both confs in a path, just do it as-needed using
composition. So using the old-fashioned ivy:retrieve and lib dirs, you'd
have:
<path id="classpath.tests.compile">
<fileset dir="${compileSrcLibDir}">
<include name="**/*.jar" />
</fileset>
<fileset dir="${compileTestsLibDir}">
<include name="**/*.jar" />
</fileset>
</path>
Or to use the more direct ivy:cachepath where ivy.src.classpath and
ivy.test.classpath are the paths you've created therein:
<path id="classpath.tests.compile">
<path refid="ivy.src.classpath" />
<path refid="ivy.test.classpath" />
</path>
On Tue, Jan 5, 2010 at 4:18 PM, Burt Leung <bu...@gmail.com> wrote:
> Hi All,
>
> This is sort of both a question and a suggestion.
>
> The team that I am on is still in the process of (starting to) adopt Ivy as
> part of our development process. One of the key concerns raised was the
> fact
> that the use of configurations "groups" can produce redundant JAR files in
> the lib folder.
>
>
> For example, I currently have configurations defined for one of our modules
> like so:
> ================
> <configurations>
> <conf name="compileSrcConf" description="Configuration for compiling
> the source code."/>
> <conf name="default" extends="compileSrcConf" description="Default
> configuration"/>
>
> <conf name="compileTestsConf" extends="compileSrcConf"
> description="Configuration for compiling the test code."/>
> <conf name="checkstyleConf" extends="compileSrcConf"
> description="Configuration for running checkstyle."/>
> </configurations>
> <dependencies>
> <dependency org="someDependencyA" name="collections" rev="1.0"
> conf="compileSrcConf->default"/>
> <dependency org="someDependencyB" name="collections" rev="1.0"
> conf="compileSrcConf->default"/>
> <dependency org="someDependencyC" name="collections" rev="1.0"
> conf="compileSrcConf->default"/>
> <dependency org="someDependencyD" name="collections" rev="1.0"
> conf="compileSrcConf->default"/>
> <dependency org="someDependencyE" name="collections" rev="1.0"
> conf="compileSrcConf->default"/>
> <dependency org="someDependencyF" name="collections" rev="1.0"
> conf="compileSrcConf->default"/>
>
> <dependency org="checkstyle" name="checkstyle" rev="4.1"
> conf="checkstyleConf->default"/>
>
> <dependency org="junit" name="junit" rev="4.1"
> conf="compileTestsConf->default"/>
> </dependencies>
> ================
>
>
> Because some of the configurations extend the compileSrcConf configuration,
> a typical Ivy retrieve will produce a lib folder that looks like the
> following:
> ================
> /lib/compileSrcConf/
> someDependencyA.jar
> someDependencyB.jar
> someDependencyC.jar
> someDependencyD.jar
> someDependencyE.jar
> someDependencyF.jar
> /lib/compileTestsConf/
> someDependencyA.jar
> someDependencyB.jar
> someDependencyC.jar
> someDependencyD.jar
> someDependencyE.jar
> someDependencyF.jar
> junit.jar
> /lib/checkstyleConf/
> someDependencyA.jar
> someDependencyB.jar
> someDependencyC.jar
> someDependencyD.jar
> someDependencyE.jar
> someDependencyF.jar
> checkstyle.jar
> ================
>
>
> We are considering removing the "extends" attribute from each configuration
> declaration so that there would not be this redundency of JAR files. We'd
> of
> course have to setup composite paths/pathelements in the ANT build file so
> that this would work. For example, an excerpt for the ANT build file:
> ================
> <property name="compileSrcLibDir"
> value="${ivyResolveLibDir}/compileSrcConf"/>
> <property name="compileTestsLibDir"
> value="${ivyResolveLibDir}/compileTestsConf"/>
> <property name="checkstyleLibDir"
> value="${ivyResolveLibDir}/checkstyleConf"/>
> ...
> <!-- The classpath used to compile the module. -->
> <path id="classpath.module.compile">
> <fileset dir="${compileSrcLibDir}">
> <include name="**/*.jar" />
> </fileset>
> </path>
>
> <!-- The classpath used to compile the tests -->
> <path id="classpath.tests.compile">
> <fileset dir="${compileTestsLibDir}">
> <include name="**/*.jar" />
> </fileset>
>
> <path refid="classpath.module.compile" /> ***<== workaround for
> not using "extends" feature in Ivy configuration.
> </path>
>
>
> <!-- The classpath used to compile the tests -->
> <path id="classpath.checkstyle">
> <fileset dir="${compileTestsLibDir}">
> <include name="**/*.jar" />
> </fileset>
>
> <path refid="classpath.module.compile" /> ***<== workaround for not
> using "extends" feature in Ivy configuration.
> </path>
> ================
>
>
> Question: is there a good argument for ignoring/tolerating this redundancy
> of JAR files?
>
> I personally think it would be nice if Ivy setup/provided ANT path elements
> behind the scenes based on the Ivy configuration groups. As I understand
> it,
> when Ivy does a resolve/retrieve, it first downloads the JAR files to its
> cache directory on the computer (e.g. "c:/documents and
> settings/user/.ivy/"). Then it will copy the JAR files specified by the
> configuration group(s) defined in a module's ivy.xml into its lib folder
> using a pattern like:
>
>
> pattern="${ivyResolveLibDir}/[conf]/[organization]/[artifact]-[revision].[ext]"
>
> Since the JAR files are already in the Ivy cache directory I think that Ivy
> could (automatically) create ANT pathelements behind the scenes (for each
> Ivy configuration group) that could then be referenced from the ANT build
> file. This would save the "copy" step I mentioned above. Does anyone else
> think this would be an improvement?
>
> Thanks,
> Burt
>