You are viewing a plain text version of this content. The canonical link for it is here.
Posted to ivy-user@ant.apache.org by Travis Zimmerman <Tr...@bhmi.com> on 2009/09/16 15:31:03 UTC
Use of conf in a complex project
I have a situation where we are trying to build a complex project that
contains multiple modules that reside in their own branch in our SCM
system. All modules are retrieved from scm separately and do not reside
within a common directory. The deliverable for the entire project is an
.ear file, each of the modules will either be a library, an ejb or a war
that is packaged within the ear file. To make things more complex, some
of the modules are optional and may not be delivered or built.
I am attempting to solve a portion of the complexity using Ivy and the
usage of confs to pull together the dependencies and help build the ear
file and application.xml. I do not believe all of this is supported but
I would like to make sure I am going down the right path to make the
most use of Ivy. I have defined several of my own "confs" to help
achieve this goal. The first is "packaged" which implies that the
dependency needs to be added to the ear file in its lib directory. The
second is "subsystem" which states that the dependency is a module with
in my project. I also then make use of the standard confs "compile',
"master" and "test". My goal is to leave the declaration of dependencies
to the actual module that actually has that dependency. However, when
building the ear file, I need some way of discovering that one of the
modules I depend on has a dependency on a library that needs to be added
to the ear file. It is also possible that this dependency is an ejb and
should be added as its own module in the ear file.
Below is a simplified example of the situation that we have, I hope that
this makes it easier to understand.
Module A - The master has dependencies on all of the modules and knows
which other modules should be included.
depends - Module B conf - subsystem->master
depends - Module C conf - subsystem->master
depends - Module D conf - subsystem->master
depends - Module E conf - subsystem->master
artifact - myproject.ear conf - subsystem
Module B
depends - Module C conf - subsystem->master
artifact - module-b.jar conf - subsystem
Module C
depends - Module D conf - subsystem->master
depends - Module E conf - subsystem->master
depends - servlet-api conf - compile->master
depends - commons-beans conf - packaged->master
artifact - module-c.jar conf - subsystem
artifact - module-c-ui.war conf - subsystem
Module D
depends - Module E conf - subsystem->master
depends - jakarta oro conf - packaged->master
artifact - module-d.jar conf - subsystem->master
artifact - module-d-ui.war conf - subsystem->master
Module E
depends - jboss.seam conf - packaged->master
depends - testng conf - test->test
artifact - module-e.jar conf - subsystem->master
My approach is to define the dependencies using the conf and retrieve
them during the build to a local cache ( within the module directory )
by conf type. When building Module A I need to create an ear file and
add the dependencies. After performing calling the retrieve I would end
up with a local cache directory with something like the following
contents
ModueA/
.libcache/
subsystem/
/ejbs
module-e.jar
/jars
module-b.jar
module-c.jar
module-d.jar
/wars
module-d-ui.war
module-c-ui.war
packaged/
/ejbs
jboss-seam.jar
/jars
jakarta-oro.jar
commons-beans.jar
I believe with things properly copied to these directories I can
dynamically build the application.xml file and the ear file without
having to duplicate the declaration of dependencies. What is actually
happening is that all of the dependencies end up in my subsystem cache
and I no longer can determine what I need to do with the artifacts.
Does this approach look valid and am I on the correct path? Is all of
this supported; if not which portion do I need to provide and what
approach should I take to provide it?
I appreciate any and all feed back.
Thanks in advance.
Re: Use of conf in a complex project
Posted by Mitch Gitman <mg...@gmail.com>.
Travis, you write: "The base level of the subsystem calls the build of each
of its subprojects and publishes artifacts that are a merge of its
subprojects." The idea of a "merge" kinda smells bad to me.
>From the standpoint of Ivy, there is no such thing as a subsystem, even
though there may be modules that serve as the interface for a particular
subsystem that's exposed to other ones. If each subsystem is is really made
up of multiple projects, then each project should be its own Ivy module and
one should be depending on the other. The ui project for subsystem A could
depend on core for subsystem A. If subsystem B only needs the core of
subsystem A, then that should depend directly on the core.
Now, I'm sure you're saying right now, "This guy's really oversimplifying
things." But you might want to just take a step back, expunge the concepts
of "merge" and "subsystem," draw a dependency graph between Ivy modules on a
whiteboard and try to use different colored pens to trace different conf
paths.
Most important, it sounds like you could accomplish a lot by giving certain
modules two special confs: "with-seam" and "without-seam". This way you can
make a single, discrete decision out of the picking up of all those
third-party libraries.
On Wed, Sep 16, 2009 at 8:16 AM, Travis Zimmerman <Travis.Zimmerman@bhmi.com
> wrote:
> Thanks for the reply Mitch.
>
> No, the subsystems and the master are not unrelated. The master may have
> code the extends any of the subsystems and the subsystems themselves
> have dependencies on other subsystems.
>
> I think I may have over simplified the situation. Each of the subsystems
> I denoted below are actually multiproject subsystems. Each (or most)
> have a core or service level project and a ui or war project. The base
> level of the subsystem calls the build of each of its subprojects and
> publishes artifacts that are a merge of its subprojects. Each of the
> subprojects publish its artifacts so that inter subsystem dependencies
> are met. The master build really wants the artifact published at the
> subsystem level and the "packaged" dependencies of the service level
> subprojects. The dependencies of the ui projects are in the war itself
> so I do not need to worry about these.
>
> The dependencies that are "subsytem" are configured to be transitive.
> All other dependencies are configured to not be transitive, this way I
> do not have to worry about carrying forward all of the dependencies seam
> has.
>
> I actually had a conf misconfigured, so I am a little further along.
> Where I am at the present is that in the master build cache directory, I
> have all dependencies as expected. They are all within the cache
> directory for the conf "subsystem" which I think will be fine. However,
> I also now have jar and war files for the subprojects within a
> subsystem. The contents for these files are published in the artifact
> for the subsystem. I don't think I want to turn off transitive as I
> would then be missing the third party libraries that are dependencies of
> the subprojects. I am not sure how to define the subsystem ivy file so
> that it is a composite of the subprojects and the thirdparty
> dependencies of each without duplicating the subproject dependencies in
> the subsystem.
>
> Any thoughts?
>
>
>
RE: Use of conf in a complex project
Posted by Travis Zimmerman <Tr...@bhmi.com>.
Thanks for the reply Mitch.
No, the subsystems and the master are not unrelated. The master may have
code the extends any of the subsystems and the subsystems themselves
have dependencies on other subsystems.
I think I may have over simplified the situation. Each of the subsystems
I denoted below are actually multiproject subsystems. Each (or most)
have a core or service level project and a ui or war project. The base
level of the subsystem calls the build of each of its subprojects and
publishes artifacts that are a merge of its subprojects. Each of the
subprojects publish its artifacts so that inter subsystem dependencies
are met. The master build really wants the artifact published at the
subsystem level and the "packaged" dependencies of the service level
subprojects. The dependencies of the ui projects are in the war itself
so I do not need to worry about these.
The dependencies that are "subsytem" are configured to be transitive.
All other dependencies are configured to not be transitive, this way I
do not have to worry about carrying forward all of the dependencies seam
has.
I actually had a conf misconfigured, so I am a little further along.
Where I am at the present is that in the master build cache directory, I
have all dependencies as expected. They are all within the cache
directory for the conf "subsystem" which I think will be fine. However,
I also now have jar and war files for the subprojects within a
subsystem. The contents for these files are published in the artifact
for the subsystem. I don't think I want to turn off transitive as I
would then be missing the third party libraries that are dependencies of
the subprojects. I am not sure how to define the subsystem ivy file so
that it is a composite of the subprojects and the thirdparty
dependencies of each without duplicating the subproject dependencies in
the subsystem.
Any thoughts?
-----Original Message-----
From: Mitch Gitman [mailto:mgitman@gmail.com]
Sent: Wednesday, September 16, 2009 9:24 AM
To: ivy-user@ant.apache.org
Subject: Re: Use of conf in a complex project
Travis, is it true that your master and subsystem confs are unrelated to
each other, i.e. neither extends the other? If that's the case, you
might have a simplified ivy.xml like this. Note that this is the ivy.xml
for an EAR project but it could just as well apply to a WAR or JAR
project:
<configurations>
<conf name="master" />
<conf name="subsystem" />
</configurations>
<publications>
<artifact name="enterprise-app" type="ear" conf="master" />
</publications>
<dependencies>
<dependency org="com.foo" name="bar" rev="1.0"
conf="subsystem->master"
/>
</dependencies>
What's happening here is that the publication of the artifact is only
happening without its transitive dependencies. You're publishing
enterprise-app.ear in the master conf, but because the dependency is
only being picked up in the subsystem conf, never the twain shall meet.
This may not be such a problem with the root, most-dependent project,
the EAR, but if it's dependency modules are only publishing themselves
in master but only picking up their own dependencies in subsystem,
you're cutting off transitivity one level at a time.
Why not make your dependencies' confs be something like
"master->master", call the ivy:cachefileset Ant task when you need to
assemble all the needed JARs, and then see what shows up when you go to
copy that fileset to your desired destination?
(Actually, the "master" scope in Maven, I believe, is designed just for
this--to publish a project's artifacts without its dependencies. So in a
way you're using the word "master" correctly, although don't ask me what
the value of that is in Maven.)
On Wed, Sep 16, 2009 at 6:31 AM, Travis Zimmerman
<Travis.Zimmerman@bhmi.com
> wrote:
>
> I have a situation where we are trying to build a complex project that
> contains multiple modules that reside in their own branch in our SCM
> system. All modules are retrieved from scm separately and do not
> reside within a common directory. The deliverable for the entire
> project is an .ear file, each of the modules will either be a library,
> an ejb or a war that is packaged within the ear file. To make things
> more complex, some of the modules are optional and may not be
delivered or built.
>
> I am attempting to solve a portion of the complexity using Ivy and the
> usage of confs to pull together the dependencies and help build the
> ear file and application.xml. I do not believe all of this is
> supported but I would like to make sure I am going down the right path
> to make the most use of Ivy. I have defined several of my own "confs"
> to help achieve this goal. The first is "packaged" which implies that
> the dependency needs to be added to the ear file in its lib directory.
> The second is "subsystem" which states that the dependency is a module
> with in my project. I also then make use of the standard confs
> "compile', "master" and "test". My goal is to leave the declaration of
> dependencies to the actual module that actually has that dependency.
> However, when building the ear file, I need some way of discovering
> that one of the modules I depend on has a dependency on a library that
> needs to be added to the ear file. It is also possible that this
> dependency is an ejb and should be added as its own module in the ear
file.
>
>
> Below is a simplified example of the situation that we have, I hope
> that this makes it easier to understand.
>
> Module A - The master has dependencies on all of the modules and knows
> which other modules should be included.
> depends - Module B conf - subsystem->master
> depends - Module C conf - subsystem->master
> depends - Module D conf - subsystem->master
> depends - Module E conf - subsystem->master
>
> artifact - myproject.ear conf - subsystem
>
> Module B
> depends - Module C conf - subsystem->master
>
> artifact - module-b.jar conf - subsystem
>
> Module C
> depends - Module D conf - subsystem->master
> depends - Module E conf - subsystem->master
> depends - servlet-api conf - compile->master
> depends - commons-beans conf - packaged->master
>
> artifact - module-c.jar conf - subsystem
> artifact - module-c-ui.war conf - subsystem
>
> Module D
> depends - Module E conf - subsystem->master
> depends - jakarta oro conf - packaged->master
>
> artifact - module-d.jar conf - subsystem->master
> artifact - module-d-ui.war conf - subsystem->master
>
> Module E
> depends - jboss.seam conf - packaged->master
> depends - testng conf - test->test
>
> artifact - module-e.jar conf - subsystem->master
>
>
> My approach is to define the dependencies using the conf and retrieve
> them during the build to a local cache ( within the module directory )
> by conf type. When building Module A I need to create an ear file and
> add the dependencies. After performing calling the retrieve I would
> end up with a local cache directory with something like the following
> contents
>
> ModueA/
> .libcache/
> subsystem/
> /ejbs
> module-e.jar
> /jars
> module-b.jar
> module-c.jar
> module-d.jar
> /wars
> module-d-ui.war
> module-c-ui.war
> packaged/
> /ejbs
> jboss-seam.jar
> /jars
> jakarta-oro.jar
> commons-beans.jar
>
>
> I believe with things properly copied to these directories I can
> dynamically build the application.xml file and the ear file without
> having to duplicate the declaration of dependencies. What is actually
> happening is that all of the dependencies end up in my subsystem cache
> and I no longer can determine what I need to do with the artifacts.
>
> Does this approach look valid and am I on the correct path? Is all of
> this supported; if not which portion do I need to provide and what
> approach should I take to provide it?
>
>
> I appreciate any and all feed back.
>
> Thanks in advance.
>
>
Re: Use of conf in a complex project
Posted by Mitch Gitman <mg...@gmail.com>.
Travis, is it true that your master and subsystem confs are unrelated to
each other, i.e. neither extends the other? If that's the case, you might
have a simplified ivy.xml like this. Note that this is the ivy.xml for an
EAR project but it could just as well apply to a WAR or JAR project:
<configurations>
<conf name="master" />
<conf name="subsystem" />
</configurations>
<publications>
<artifact name="enterprise-app" type="ear" conf="master" />
</publications>
<dependencies>
<dependency org="com.foo" name="bar" rev="1.0" conf="subsystem->master"
/>
</dependencies>
What's happening here is that the publication of the artifact is only
happening without its transitive dependencies. You're publishing
enterprise-app.ear in the master conf, but because the dependency is only
being picked up in the subsystem conf, never the twain shall meet. This may
not be such a problem with the root, most-dependent project, the EAR, but if
it's dependency modules are only publishing themselves in master but only
picking up their own dependencies in subsystem, you're cutting off
transitivity one level at a time.
Why not make your dependencies' confs be something like "master->master",
call the ivy:cachefileset Ant task when you need to assemble all the needed
JARs, and then see what shows up when you go to copy that fileset to your
desired destination?
(Actually, the "master" scope in Maven, I believe, is designed just for
this--to publish a project's artifacts without its dependencies. So in a way
you're using the word "master" correctly, although don't ask me what the
value of that is in Maven.)
On Wed, Sep 16, 2009 at 6:31 AM, Travis Zimmerman <Travis.Zimmerman@bhmi.com
> wrote:
>
> I have a situation where we are trying to build a complex project that
> contains multiple modules that reside in their own branch in our SCM
> system. All modules are retrieved from scm separately and do not reside
> within a common directory. The deliverable for the entire project is an
> .ear file, each of the modules will either be a library, an ejb or a war
> that is packaged within the ear file. To make things more complex, some
> of the modules are optional and may not be delivered or built.
>
> I am attempting to solve a portion of the complexity using Ivy and the
> usage of confs to pull together the dependencies and help build the ear
> file and application.xml. I do not believe all of this is supported but
> I would like to make sure I am going down the right path to make the
> most use of Ivy. I have defined several of my own "confs" to help
> achieve this goal. The first is "packaged" which implies that the
> dependency needs to be added to the ear file in its lib directory. The
> second is "subsystem" which states that the dependency is a module with
> in my project. I also then make use of the standard confs "compile',
> "master" and "test". My goal is to leave the declaration of dependencies
> to the actual module that actually has that dependency. However, when
> building the ear file, I need some way of discovering that one of the
> modules I depend on has a dependency on a library that needs to be added
> to the ear file. It is also possible that this dependency is an ejb and
> should be added as its own module in the ear file.
>
>
> Below is a simplified example of the situation that we have, I hope that
> this makes it easier to understand.
>
> Module A - The master has dependencies on all of the modules and knows
> which other modules should be included.
> depends - Module B conf - subsystem->master
> depends - Module C conf - subsystem->master
> depends - Module D conf - subsystem->master
> depends - Module E conf - subsystem->master
>
> artifact - myproject.ear conf - subsystem
>
> Module B
> depends - Module C conf - subsystem->master
>
> artifact - module-b.jar conf - subsystem
>
> Module C
> depends - Module D conf - subsystem->master
> depends - Module E conf - subsystem->master
> depends - servlet-api conf - compile->master
> depends - commons-beans conf - packaged->master
>
> artifact - module-c.jar conf - subsystem
> artifact - module-c-ui.war conf - subsystem
>
> Module D
> depends - Module E conf - subsystem->master
> depends - jakarta oro conf - packaged->master
>
> artifact - module-d.jar conf - subsystem->master
> artifact - module-d-ui.war conf - subsystem->master
>
> Module E
> depends - jboss.seam conf - packaged->master
> depends - testng conf - test->test
>
> artifact - module-e.jar conf - subsystem->master
>
>
> My approach is to define the dependencies using the conf and retrieve
> them during the build to a local cache ( within the module directory )
> by conf type. When building Module A I need to create an ear file and
> add the dependencies. After performing calling the retrieve I would end
> up with a local cache directory with something like the following
> contents
>
> ModueA/
> .libcache/
> subsystem/
> /ejbs
> module-e.jar
> /jars
> module-b.jar
> module-c.jar
> module-d.jar
> /wars
> module-d-ui.war
> module-c-ui.war
> packaged/
> /ejbs
> jboss-seam.jar
> /jars
> jakarta-oro.jar
> commons-beans.jar
>
>
> I believe with things properly copied to these directories I can
> dynamically build the application.xml file and the ear file without
> having to duplicate the declaration of dependencies. What is actually
> happening is that all of the dependencies end up in my subsystem cache
> and I no longer can determine what I need to do with the artifacts.
>
> Does this approach look valid and am I on the correct path? Is all of
> this supported; if not which portion do I need to provide and what
> approach should I take to provide it?
>
>
> I appreciate any and all feed back.
>
> Thanks in advance.
>
>