You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@maven.apache.org by Christian Clausen <ch...@tietoenator.com> on 2003/02/06 13:24:28 UTC
Dependencies and local repository clean-up
Hi,
I have noticed the following behaviour concerning maven dependencies:
Suppose a dependency named D is in the remote repository (think of D as
a snapshot jar file matching the pattern "*-SNAPSHOT.jar"). The first
time a goal of a project depending on D is attained, the artifact is
downloaded to the local repository. Suppose now that a new version of D
is deployed to the remote repository, and that the version of D is kept
unchanged (hence the final name including version is unchanged). When
the goal of the depending project is attained now, the new version of D
is NOT installed in the local repository. Consequently, we are still
using the old version of D.
How do I avoid this situation? Is there a standard way of flushing the
local repository (per groupId)? Or is it somehow possible to have maven
use an HTTP header to check the remote file date against the local file
date?
An obvious workaround is to code a simple goal which removes some or all
files of a given groupId from the local repository.
The described situation is not so common if you are developing a single
project depending on 3rd party or own production versions (x.y.z) since
such files are never modified. But during development in a larger
project consisting of several maven projects, I think it is good to be
able to share "newest deployed versions" without having to be explicit
about the "z" in x.y.z.
Regards,
Christian
Re: Dependencies and local repository clean-up
Posted by Brian Ewins <Br...@btinternet.com>.
Christian Clausen wrote:
> Brian Ewins wrote:
>> Christian Clausen wrote:
>>> Another issue is how to manage dependencies in a production branch of
>>> a group of dependent maven projects. Obviously, versions in a
>>> production branch must be "real" (x.y.z). [... if you change a
>>> utility version ...]
>>> all J2EE modules depending on the utility must be patched and
>>> released/deployed.
>> That's exactly right, and with good reason. If you change the version
>> of a utility, how do you know that the things which depended upon the
>> previous version still work? Being explicit about dependencies does
>> mean more work, but it pays off in the long run.
> OK, so being forced to update all dependencies can be seen as a "patch
> certification procedure". Is there a way to get a single overview of a
> potentially large dependency tree? (I mean, in case module A depends on
> module B which depends on the changed utility, the certification
> procedure must begin with module B and then A.)
Well, yes and no. Think of it this way: if you have B.war depending on
A-1.0.jar, and you update A-1.0.jar to A-1.1.jar, have you changed the
dependency tree? The answer is no - B.war still depends on A-1.0.jar
because thats the explicity dependency you've written down. Maven won't
try to figure out whether you mean for B to be upgraded or not.
On the other hand, if you have a collection of interdependent projects,
can you figure out which ones might need changed if one of them is
updated? The answer to this is more likely yes, since the Reactor has to
this to figure out the correct order to build projects. I don't know if
the reactor produces a dependency cross-reference report for all the
projects at the moment, but projects would need updated in exactly the
same order as the Reactor builds them.
-Baz
Re: Dependencies and local repository clean-up
Posted by Christian Clausen <ch...@tietoenator.com>.
Brian Ewins wrote:
>
>
> Christian Clausen wrote:
>
>> I'm in doubt about how snapshot versions fit into the release process
>> described in:
>> http://jakarta.apache.org/turbine/maven/development/release-process.html.
>>
>>
>> This document mentions meaningful versions like "3.0-dev", "3.0-b1",
>> and "3.0-rc1". Are such versions meant to be used as a project's
>> currentVersion, and the project can be deployed in a special snapshot
>> way such the deployed artifact's version is "3.0-SNAPSHOT"? Or is
>> "3.0-SNAPSHOT" meant to be used directly as currentVersion?
>
> The instructions on that page are not current, but are advocating
> developing with a SNAPSHOT most of the time. It should read '-SNAPSHOT'
> everywhere it currently says '-dev'. You can set a projects version to
> include 'SNAPSHOT' by editing the project.xml, or by running 'maven
> jar:install-snapshot', or 'maven jar:deploy-snapshot', the
> recommendation is to do it by changing the project.xml.
>
>> Another issue is how to manage dependencies in a production branch of
>> a group of dependent maven projects. Obviously, versions in a
>> production branch must be "real" (x.y.z). [... if you change a utility
>> version ...]
>> all J2EE modules depending on the utility must be patched and
>> released/deployed.
>
> That's exactly right, and with good reason. If you change the version of
> a utility, how do you know that the things which depended upon the
> previous version still work? Being explicit about dependencies does mean
> more work, but it pays off in the long run.
>
OK, so being forced to update all dependencies can be seen as a "patch
certification procedure". Is there a way to get a single overview of a
potentially large dependency tree? (I mean, in case module A depends on
module B which depends on the changed utility, the certification
procedure must begin with module B and then A.)
>> How do I come around this problem? One alternative, I could imagine,
>> is to strip artifact versions from jar names in the manifest classpath
>> and strip versions from files when included in EAR or WAR files. I
>> admit that this stripping is a bit un-mavenish.
>>
>> How do you see this?
>
> Maven should write the Class-Path: entry for you, but you should be
> explicit in your project.xml about the things you depend upon. In other
> words, maven is making it easier for you to follow a disciplined
> approach that would otherwise be a right royal pain in the bahookie.
>
>> What is, by the way, the benefit of having version numbers in artifact
>> names? Of course, it is easy to see the version; on the other hand
>> this information is already in the artifact's manifest file.
>
>
> - its better for support. Support teams and customers don't understand
> manifests, but ask them what the file name is and you'll get a straight
> answer.
> - most manifests created without maven don't include this information.
> That's why the fingerprint.jsp that I wrote in apache axis prints file
> sizes instead of dipping into the jars to check versions - the
> information was useless.
> - most tools used during the build process know about files, not jar
> manifests. E.g. a tool to copy a file from one place to another. By
> putting version info into the filename, you let these tools help you
> check if your build is consistent.
> - it makes errors easy to spot just looking at a directory listing.
>
> >
> In Maven, the
>
>> artifact name is also used for ensuring unique referencing of the
>> remote repository. If artifact names did not include the version
>> number, then Maven repositories should be organized with an additional
>> artifact version layer like e.g. "groupId - version - type".
>
>
> Yes, the repository would be better with a metadata layer, but there
> isn't one as yet. Lots of people (including myself) have suggested this,
> and other changes to the repository design, but the response from the
> maven team is quite rightly 'send us the patches'. The current repo
> design is about the simplest thing that gets the job done, anything else
> is gravy.
>
> Cheers,
> Baz
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: turbine-maven-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: turbine-maven-user-help@jakarta.apache.org
>
>
>
--
Christian Clausen, M.Sc.
TietoEnator Financial Solutions A/S
Ved Lunden 12
DK-8230 Åbyhøj
Denmark
Phone: +45 7027 6400
Direct: +45 8746 6460
Fax: +45 7027 6440
Email: christian.clausen@tietoenator.com
Internet: www.tietoenator.com
--
Re: Dependencies and local repository clean-up
Posted by Brian Ewins <Br...@btinternet.com>.
Christian Clausen wrote:
> I'm in doubt about how snapshot versions fit into the release process
> described in:
> http://jakarta.apache.org/turbine/maven/development/release-process.html.
>
> This document mentions meaningful versions like "3.0-dev", "3.0-b1", and
> "3.0-rc1". Are such versions meant to be used as a project's
> currentVersion, and the project can be deployed in a special snapshot
> way such the deployed artifact's version is "3.0-SNAPSHOT"? Or is
> "3.0-SNAPSHOT" meant to be used directly as currentVersion?
The instructions on that page are not current, but are advocating
developing with a SNAPSHOT most of the time. It should read '-SNAPSHOT'
everywhere it currently says '-dev'. You can set a projects version to
include 'SNAPSHOT' by editing the project.xml, or by running 'maven
jar:install-snapshot', or 'maven jar:deploy-snapshot', the
recommendation is to do it by changing the project.xml.
> Another issue is how to manage dependencies in a production branch of a
> group of dependent maven projects. Obviously, versions in a production
> branch must be "real" (x.y.z). [... if you change a utility version ...]
> all J2EE modules depending on the utility must be patched and
> released/deployed.
That's exactly right, and with good reason. If you change the version of
a utility, how do you know that the things which depended upon the
previous version still work? Being explicit about dependencies does mean
more work, but it pays off in the long run.
> How do I come around this problem? One alternative, I could imagine, is
> to strip artifact versions from jar names in the manifest classpath and
> strip versions from files when included in EAR or WAR files. I admit
> that this stripping is a bit un-mavenish.
>
> How do you see this?
Maven should write the Class-Path: entry for you, but you should be
explicit in your project.xml about the things you depend upon. In other
words, maven is making it easier for you to follow a disciplined
approach that would otherwise be a right royal pain in the bahookie.
> What is, by the way, the benefit of having version numbers in artifact
> names? Of course, it is easy to see the version; on the other hand this
> information is already in the artifact's manifest file.
- its better for support. Support teams and customers don't understand
manifests, but ask them what the file name is and you'll get a straight
answer.
- most manifests created without maven don't include this information.
That's why the fingerprint.jsp that I wrote in apache axis prints file
sizes instead of dipping into the jars to check versions - the
information was useless.
- most tools used during the build process know about files, not jar
manifests. E.g. a tool to copy a file from one place to another. By
putting version info into the filename, you let these tools help you
check if your build is consistent.
- it makes errors easy to spot just looking at a directory listing.
>
In Maven, the
> artifact name is also used for ensuring unique referencing of the remote
> repository. If artifact names did not include the version number, then
> Maven repositories should be organized with an additional artifact
> version layer like e.g. "groupId - version - type".
Yes, the repository would be better with a metadata layer, but there
isn't one as yet. Lots of people (including myself) have suggested this,
and other changes to the repository design, but the response from the
maven team is quite rightly 'send us the patches'. The current repo
design is about the simplest thing that gets the job done, anything else
is gravy.
Cheers,
Baz
Re: Dependencies and local repository clean-up
Posted by Christian Clausen <ch...@tietoenator.com>.
Thanks for your explanation - the SNAPSHOT feature is really nice! Also,
your statement concerning repeatability is important: Never replacing
non-snapshot artifacts in the remote repository could even be coded in
the deploy plugin.
I'm in doubt about how snapshot versions fit into the release process
described in:
http://jakarta.apache.org/turbine/maven/development/release-process.html.
This document mentions meaningful versions like "3.0-dev", "3.0-b1", and
"3.0-rc1". Are such versions meant to be used as a project's
currentVersion, and the project can be deployed in a special snapshot
way such the deployed artifact's version is "3.0-SNAPSHOT"? Or is
"3.0-SNAPSHOT" meant to be used directly as currentVersion?
Another issue is how to manage dependencies in a production branch of a
group of dependent maven projects. Obviously, versions in a production
branch must be "real" (x.y.z). Most J2EE servers require dependencies to
be listed in the manifest classpath. Consequently, if a utility is
patched, all J2EE modules depending on this utility must be repackaged
because of the changed jar name. But when a J2EE module is repackaged,
it should (correct me if I'm wrong) also be redeployed to the remote
repository and hence have a new version number assigned first. This
means that all J2EE modules depending on the utility must be patched and
released/deployed.
How do I come around this problem? One alternative, I could imagine, is
to strip artifact versions from jar names in the manifest classpath and
strip versions from files when included in EAR or WAR files. I admit
that this stripping is a bit un-mavenish.
How do you see this?
What is, by the way, the benefit of having version numbers in artifact
names? Of course, it is easy to see the version; on the other hand this
information is already in the artifact's manifest file. In Maven, the
artifact name is also used for ensuring unique referencing of the remote
repository. If artifact names did not include the version number, then
Maven repositories should be organized with an additional artifact
version layer like e.g. "groupId - version - type".
Regards,
Christian
Brian Ewins wrote:
>
>
> Christian Clausen wrote:
>
>> Hi,
>>
>> I have noticed the following behaviour concerning maven dependencies:
>>
>> [...] The first time a goal of a project depending on D is attained,
>> the artifact is downloaded to the local repository. Suppose now that a
>> new version of D is deployed to the remote repository, [...] the new
>> version of D
>> is NOT installed in the local repository. Consequently, we are still
>> using the old version of D.
>
>
> This is not the case with snapshot jars - they are always downloaded
> again if you are online.
>
> The point of version numbers is to make a build repeatable. If people
> use a /real/ version number, then replace the artifact, they are
> shooting themselves in the foot - builds are no longer repeatable. The
> point of SNAPSHOTS is to be able to ask for something where the name
> stays the same but you are willing to sacrifice repeatability - ie
> during development of related projects.
>
>> ...during development in a larger
>> project consisting of several maven projects, I think it is good to be
>> able to share "newest deployed versions" without having to be explicit
>> about the "z" in x.y.z.
>
>
> Try building, and depending on, artifact-x.y-SNAPSHOT.jar. (ie version
> x.y-SNAPSHOT instead of x.y.z) This should work exactly as you want, in
> recent b8 builds. This even allows you to build multiple 'working
> versions' of a project, eg tomcat-3.3-SNAPSHOT.jar,
> tomcat-4.1-SNAPSHOT.jar can coexist.
>
> I can see one circumstance where having an unique name instead of
> SNAPSHOT for non-release builds is useful - thats when you're doing
> nightly builds, or otherwise timestamping jars, and you want to keep
> (eg) the last 5 nights builds around (note that keeping builds with
> unique names is for repeatability again). In this case, why not copy the
> nightly to the SNAPSHOT name in a postGoal? That way you have a SNAPSHOT
> thats consistent with the nightly, but you can keep using the same name
> if you don't care.
>
> The SNAPSHOT convention neatly avoids having to decide what the 'latest'
> version of an artifact is (eg I have tomcat 3.2 - is tomcat 3.3 or
> tomcat 4.1 the latest? Answering 4.1 might break your code, and
> timestamps won't help you), and avoids attempting to do wildcards over
> the repository-protocol-of-the-day (eg tomcat-3.* - which still leaves
> you having to make decisions on the meaning of version numbers.)
>
> Cheers,
> -Baz
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: turbine-maven-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: turbine-maven-user-help@jakarta.apache.org
>
>
>
Re: Dependencies and local repository clean-up
Posted by Brian Ewins <Br...@btinternet.com>.
Christian Clausen wrote:
> Hi,
>
> I have noticed the following behaviour concerning maven dependencies:
>
> [...] The first
> time a goal of a project depending on D is attained, the artifact is
> downloaded to the local repository. Suppose now that a new version of D
> is deployed to the remote repository, [...] the new version of D
> is NOT installed in the local repository. Consequently, we are still
> using the old version of D.
This is not the case with snapshot jars - they are always downloaded
again if you are online.
The point of version numbers is to make a build repeatable. If people
use a /real/ version number, then replace the artifact, they are
shooting themselves in the foot - builds are no longer repeatable. The
point of SNAPSHOTS is to be able to ask for something where the name
stays the same but you are willing to sacrifice repeatability - ie
during development of related projects.
> ...during development in a larger
> project consisting of several maven projects, I think it is good to be
> able to share "newest deployed versions" without having to be explicit
> about the "z" in x.y.z.
Try building, and depending on, artifact-x.y-SNAPSHOT.jar. (ie version
x.y-SNAPSHOT instead of x.y.z) This should work exactly as you want, in
recent b8 builds. This even allows you to build multiple 'working
versions' of a project, eg tomcat-3.3-SNAPSHOT.jar,
tomcat-4.1-SNAPSHOT.jar can coexist.
I can see one circumstance where having an unique name instead of
SNAPSHOT for non-release builds is useful - thats when you're doing
nightly builds, or otherwise timestamping jars, and you want to keep
(eg) the last 5 nights builds around (note that keeping builds with
unique names is for repeatability again). In this case, why not copy the
nightly to the SNAPSHOT name in a postGoal? That way you have a SNAPSHOT
thats consistent with the nightly, but you can keep using the same name
if you don't care.
The SNAPSHOT convention neatly avoids having to decide what the 'latest'
version of an artifact is (eg I have tomcat 3.2 - is tomcat 3.3 or
tomcat 4.1 the latest? Answering 4.1 might break your code, and
timestamps won't help you), and avoids attempting to do wildcards over
the repository-protocol-of-the-day (eg tomcat-3.* - which still leaves
you having to make decisions on the meaning of version numbers.)
Cheers,
-Baz