runtime dependencies and cascaded downloads

[John Casey <jd...@00java.com>]

In the past I've seen some traffic on the developer's list regarding the
resolution of transitive runtime dependencies for a particular project,
and I was wondering if there were any solid plans/thoughts/etc. on how
this is to be done?  

The project I'm working on is a set of components with a common source
tree and branding, etc. but which have definite interdependencies. 
These components each have a project.xml, and each build into their
respective jar files.  

The sticking point is that while the first order dependencies listed in
the project.xml are adequate for building a particular jar, the
transitive dependencies must also be satisfied in any distribution I put
together, before the code will actually run.  This means that the dist
goals must download/copy the first-order dependencies (the "build"
dependencies), PLUS download/copy any transitive dependencies which the
first-order deps in turn depend on at runtime (the "runtime"
dependencies).  

Clearly, this is frought with a certain peril, since one could easily
find himself d/l'ing the entire internet to create one dist, but with
proper care and planning, this type of architecture provides for really
good flexibility...

Does anyone have any thoughts?

Thanks in advance,
John

Re: runtime dependencies and cascaded downloads

[Jason van Zyl <ja...@zenplex.com>]

On Wed, 2003-02-26 at 01:01, John Casey wrote:
> In the past I've seen some traffic on the developer's list regarding the
> resolution of transitive runtime dependencies for a particular project,
> and I was wondering if there were any solid plans/thoughts/etc. on how
> this is to be done?  
> 
> The project I'm working on is a set of components with a common source
> tree and branding, etc. but which have definite interdependencies. 
> These components each have a project.xml, and each build into their
> respective jar files.  
> 
> The sticking point is that while the first order dependencies listed in
> the project.xml are adequate for building a particular jar, the
> transitive dependencies must also be satisfied in any distribution I put
> together, before the code will actually run.  This means that the dist
> goals must download/copy the first-order dependencies (the "build"
> dependencies), PLUS download/copy any transitive dependencies which the
> first-order deps in turn depend on at runtime (the "runtime"
> dependencies).  
> 
> Clearly, this is frought with a certain peril, since one could easily
> find himself d/l'ing the entire internet to create one dist, but with
> proper care and planning, this type of architecture provides for really
> good flexibility...
> 
> Does anyone have any thoughts?

Been discussed and has been noted that this really can't start until we
start getting POMs uploaded to the repository. This won't be easy until
the admin app is in place. The start of building distributions/runtimes
with transitive dependencies is in the plexus plugin which walks the
POMs basically and finds all the dependencies for a project. Eventually
a shared database will be created with all sorts of useful information
like project dependencies, URLs (for dependency display) and who knows
what else. The first step is getting the information we need.

> Thanks in advance,
> John
> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: turbine-maven-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: turbine-maven-user-help@jakarta.apache.org
-- 
jvz.

Jason van Zyl
jason@zenplex.com
http://tambora.zenplex.org

In short, man creates for himself a new religion of a rational
and technical order to justify his work and to be justified in it.
  
  -- Jacques Ellul, The Technological Society

Re: runtime dependencies and cascaded downloads

[Brian Ewins <Br...@btinternet.com>]

Downloading transitive runtime dependencies isn't as good an idea as you 
think. Examples:

- project A depends on xerces-2.2.1.jar. project B depends on 
xerces-2.3.jar. I build project C which depends on A and B. I end up 
with two copies of xerces in my .war and nothing quite works as expected.
- [ ok so we make it figure out which xerces is more recent... ] project 
A depends on xerces-2.2.1.jar. project B depends on crimson-2.0.jar. I 
end up with two xml parsers in my .war and nothing quite works as expected.
- getting the most recent implementation of an API doesnt help - A may 
depend /specifically/ on xerces, B may depend specifically on crimson. 
This is hard to spot as (for example) A may depend on a SAX property 
supported by xerces but not crimson, with no explicit mention of xerces 
anywhere in the code.
- Final example. Project A depends on xerces-2.2.1.jar. I build my 
project based on project A and it works ok. Someone points out to dIon 
that this version of project A really depends on xerces-3 for some 
features - the old metadata contained typos. The metadata is corrected, 
but xerces-3 is incompatible with my project and I can no longer repeat 
old builds.

I've used the xml parsers here because its something most people will 
have come across as being a problem at some point. However these 
problems are common to many pluggable APIs. In the long run its better 
to have explicit dependencies, it makes your builds repeatable.

However, there may be a better way to deal with transitive dependencies. 
If there was a maven plugin which you could use to write all transitive 
dependencies into your project.xml, so that you could then edit them 
down into an explicit list, that would be tremendously useful. ie you'd 
do something like this:

 > maven -f project.xml java:jar
Downloading A ....
Downloading B ....
BUILD FAILED: cannot find org.apache.xerces.BigBallOMud
 > maven -f project.xml dependencies:transitive
[whirr whirr click]
 > maven -f project.xml java:jar
Downloading xerces-2.2.1.jar .....
Downloading xerces-2.3.jar .....
^C [whoops!]
 > edit project.xml
[remove extra xerces dependency]
 > maven -f project.xml java:jar
BUILD SUCCESSFUL

-Baz

John Casey wrote:
> In the past I've seen some traffic on the developer's list regarding the
> resolution of transitive runtime dependencies for a particular project,
> and I was wondering if there were any solid plans/thoughts/etc. on how
> this is to be done?  
> 
> The project I'm working on is a set of components with a common source
> tree and branding, etc. but which have definite interdependencies. 
> These components each have a project.xml, and each build into their
> respective jar files.  
> 
> The sticking point is that while the first order dependencies listed in
> the project.xml are adequate for building a particular jar, the
> transitive dependencies must also be satisfied in any distribution I put
> together, before the code will actually run.  This means that the dist
> goals must download/copy the first-order dependencies (the "build"
> dependencies), PLUS download/copy any transitive dependencies which the
> first-order deps in turn depend on at runtime (the "runtime"
> dependencies).  
> 
> Clearly, this is frought with a certain peril, since one could easily
> find himself d/l'ing the entire internet to create one dist, but with
> proper care and planning, this type of architecture provides for really
> good flexibility...
> 
> Does anyone have any thoughts?
> 
> Thanks in advance,
> John
> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: turbine-maven-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: turbine-maven-user-help@jakarta.apache.org
> 
>