You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@maven.apache.org by "christofer.dutz@c-ware.de" <ch...@c-ware.de> on 2012/10/15 23:32:21 UTC
Independent module release strategies
Hi,
I am currently working on finetuning the workflows on a large application that was migrated from an Ant based build to one based upon Maven.
The build itself is running smoothly but, what I'm currently working on is getting the release workflow optimized.
The project consists of about 50 Maven artifacts. A lot of people are using this project all over the world. The client is distributed by some web-start similar solution.
The problem is whenever a bugfix-release is done, we don't want to release all modules in a new version because then all of these would have to be downloaded by the clients.
So we have a project with a lot of modules and a parent pom that configures the plugins.
Using the regular maven-release-plugin involves a lot of manual adjusting of version numbers and I would like to eliminate this.
That's why I setup the master pom to have two profiles "develop" (active by default) and "release" activated during a release. In both profiles a lot of properties are configured to be used for setting the artifact versions.
No comes the part where I was told on the dev-list that I was tempted by the dark side of the force ;-)
In my master pom, I defined one major dependencyManagement section fixing the version of each artifact to the versions in the properties. Ok ... so this is normal and this is not "dark side magic". But in order to have the parent version automatically configured the right way I wanted to have the version of the parent link configured by these properties too. Ok so maven doesn't allow this. But it seems that this is not entirely true:
If I configre the version of the artifact I want to use as parent with the same variable as I am using in the parent definition of the child module. Maven seems to work fine with that. The only thing that I was not quite satisfied with, was that the install plugin installed the raw poms into my local repo. The directory it was installed to contained the correct version so the resolution must have worked. That's why I thought this was a bug in the deploy plugin and that's why I filed an issue (which was immediately closed because I was doing bad magic) http://jira.codehaus.org/browse/MNG-5358
I attached an example project containing an example configuration demonstrating what I was doing.
What I find particularly strange is that Maven claims not to resolve properties in project.version and project.parent.version and in 90% of the cases this is true:
Let me illustrate thsi a little. Assuming I have only two projects ... one master and one module.
If I define two properties in my master pom: "my.cool.master.version" and "my.cool.alternate.master.version" and set both to the same value of "1.2-SNAPSHOT".
In szenario 1: I hard code the version of the master to "1.2-SNAPSHOT" but use the property to reference the parent from the moule ... when running a build, maven tries to download "de/mycompany/test/${my.cool.master.version}/mycoolmaster-${my.cool.master.version}.pom" --> Failure, because the property is not resolved
In szenario 2: I use the same variable for defining the masters version. This time the maven build runs fine and the parent version is correctly resolved.
In szenario 3: I use the first property to define the version of the master and the second one for referncing the parent from the module ... when running a build, maven tries to download "de/mycompany/test/${my.cool.alternate.master.version}/mycoolmaster-${my.cool.alternate.master.version}.pom" --> Failure, because the property is not resolved
So to me it looks as if there was some sort of intention behind everything and not a bug in the system as I was told on the dev-list. To me it looks like one teeniewienie loophole allowing properies in versions while closing the usage range so much that possible harm is reduced to it's absolute minimum. So it seems that my usecase seems to be the onlly one allowed. After all ... this is a problem users are begging for maven to provide a solution since maven 2.0 (When looking at the forums).
Ok ... and now to finish the loop back to my topic:
If I am doing bad sourcery ... how would I setup one maven build to allow simple releases of individual modules with individual versions?
Chris
[ C h r i s t o f e r D u t z ]
C-Ware IT-Service
Inhaber
Dipl. Inf. Christofer Dutz
Karlstraße. 104, 64285 Darmstadt
[cid:image001.gif@01CDAB2D.00AD5550]<http://www.benchpark.com/788335/kundenzufriedenheit.htm>
IT- und Systemhäuser<http://www.benchpark.com/it_und_systemhaeuser.htm>
fon: 0 61 51 / 27315 - 61
fax: 0 61 51 / 27315 - 64
mobil: 0171 / 7 444 2 33
email: christofer.dutz@c-ware.de<ma...@c-ware.de>
http://www.c-ware.de<http://www.c-ware.de/>
UStId-Nr. DE195700962
AW: Independent module release strategies
Posted by "christofer.dutz@c-ware.de" <ch...@c-ware.de>.
Well as soon as a major release is out the door, in that branch only bugfixes are added that have agreat effect on the application. So I am sure that mostly 2-3 Modules change. I know that in general updating a client with about 20mb doesn't sound that bad in our days, but imagine the load on the servers if hundreds of clients all over the worls start updating simultaneously. And there are quite some uses with analog modems ... updating them every time would mean a downtime of hours for them (I was told). So the company tries to keep the number of updated modules to a minimum.
Chris
________________________________________
Von: Wayne Fay [waynefay@gmail.com]
Gesendet: Dienstag, 16. Oktober 2012 01:26
An: Maven Users List
Betreff: Re: Independent module release strategies
> The project consists of about 50 Maven artifacts. A lot of people are
> using this project all over the world. The client is distributed by some
> web-start similar solution.
>
> The problem is whenever a bugfix-release is done, we don’t want to release
> all modules in a new version because then all of these would have to be
> downloaded by the clients.
Have you performed any analysis of previous changes to give you an
indication of the relative stability of one module vs the rest? Is it
true that most changes are happening in one or two modules while the
rest is stable -- or are changes found in most any module with no
obvious pattern?
I would expect that suggested approaches for "solving" your problem
may depend a bit on the answers to these questions.
Wayne
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Independent module release strategies
Posted by Ron Wheeler <rw...@artifact-software.com>.
On 15/10/2012 7:39 PM, Barrie Treloar wrote:
> On Tue, Oct 16, 2012 at 9:56 AM, Wayne Fay <wa...@gmail.com> wrote:
>> I would expect that suggested approaches for "solving" your problem
>> may depend a bit on the answers to these questions.
> Which come about because your software has a development lifecycle:
> Developing > Released > Maintained > End of Life
> Obviously each stage has different areas that will get changed.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
>
or
Developing > Released > Maintained > Maintained > Maintained > Maintained >...> End of Life
Developing > Released > Maintained > Maintained and Developed > Released And Maintained > Maintained >...> End of Life
I think that a lot of newcomers to the forum are so tied up in the initial development that they are not thinking about what happens when you need to support a package that has a user base for multiple releases while fixing bugs and developing the next release.
Maven's ability to identify all of the right dependencies to reliably reproduce a release done months ago is very helpful once you produce something useful that gets into production.
Careful management of tags and branches in our SCM is a priority for us.
This is perhaps more germane to the discussion on version ranges or profiles which I am pretty convinced are concepts directly related to "pure evil" or the road to hell, in a real software project.
Ron
--
Ron Wheeler
President
Artifact Software Inc
email: rwheeler@artifact-software.com
skype: ronaldmwheeler
phone: 866-970-2435, ext 102
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Independent module release strategies
Posted by Barrie Treloar <ba...@gmail.com>.
On Tue, Oct 16, 2012 at 9:56 AM, Wayne Fay <wa...@gmail.com> wrote:
> I would expect that suggested approaches for "solving" your problem
> may depend a bit on the answers to these questions.
Which come about because your software has a development lifecycle:
Developing > Released > Maintained > End of Life
Obviously each stage has different areas that will get changed.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Independent module release strategies
Posted by Wayne Fay <wa...@gmail.com>.
> The project consists of about 50 Maven artifacts. A lot of people are
> using this project all over the world. The client is distributed by some
> web-start similar solution.
>
> The problem is whenever a bugfix-release is done, we don’t want to release
> all modules in a new version because then all of these would have to be
> downloaded by the clients.
Have you performed any analysis of previous changes to give you an
indication of the relative stability of one module vs the rest? Is it
true that most changes are happening in one or two modules while the
rest is stable -- or are changes found in most any module with no
obvious pattern?
I would expect that suggested approaches for "solving" your problem
may depend a bit on the answers to these questions.
Wayne
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Independent module release strategies
Posted by Ron Wheeler <rw...@artifact-software.com>.
For what its worth:
1) We do not use the Maven release plug-in and follow what Barrie
described below - SCM tags, etc.
2) We do not use profiles.
3) By getting the libraries out of our code, we make the wars very small
which might help more in this case where a lot of people are downloading
bug fixes over slow lines. We generally do not change the provided
libraries within a minor update so they are very stable and there is no
need to download 20Mb to get 20Kb of code that changed. Things like CXF
are huge and are used everywhere in our application so it makes a big
difference in the builds if you have to include it in every war file.
Ron
On 16/10/2012 5:56 AM, Barrie Treloar wrote:
> On Tue, Oct 16, 2012 at 7:36 PM, Stephen Connolly
> <st...@gmail.com> wrote:
> [del]
>
> See what happens when your thread reaches critical mass!
> Kittens die.
>
> p.s. Thanks for an in-depth email that will be very useful for the archives.
>
> The "Maven Way" for a release process isn't really well documented
> anywhere yet, and I fear a few more kittens will die over that debate
> as well.
> Most people bump into release:process and blindly follow that path and
> there a limitations with that option.
> SCM commit/tags and mvn deploy are just as good.
> Its really important to understand what is being automated before automating it.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
>
--
Ron Wheeler
President
Artifact Software Inc
email: rwheeler@artifact-software.com
skype: ronaldmwheeler
phone: 866-970-2435, ext 102
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Independent module release strategies
Posted by Barrie Treloar <ba...@gmail.com>.
On Tue, Oct 16, 2012 at 7:36 PM, Stephen Connolly
<st...@gmail.com> wrote:
[del]
See what happens when your thread reaches critical mass!
Kittens die.
p.s. Thanks for an in-depth email that will be very useful for the archives.
The "Maven Way" for a release process isn't really well documented
anywhere yet, and I fear a few more kittens will die over that debate
as well.
Most people bump into release:process and blindly follow that path and
there a limitations with that option.
SCM commit/tags and mvn deploy are just as good.
Its really important to understand what is being automated before automating it.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
AW: Independent module release strategies
Posted by "christofer.dutz@c-ware.de" <ch...@c-ware.de>.
Well after quite a lot of digging in the code of the release plugin, I think that I will use a different strategy:
- in the release.properties or using the commandline, I am able to set individual versions of my modules and the release plugin will use them http://maven.apache.org/plugins/maven-release-plugin/examples/non-interactive-release.html which is basically what I wanted in the first place.
- In order to release only individual projects, using the advanced reactor otions of maven seems ideal: http://www.sonatype.com/people/2009/10/maven-tips-and-tricks-advanced-reactor-options/
So in order to have exactly what I wanted, all I need to do, is create a release.properties file to tell the plugin the versions and then limit the maven reactor to only the modules I want to release and use the normal "release:prepare" and "release:perform" targets.
As creating the properties and commandline arguments I a little uncomfortable, I'm currently wokring on a jenking plugin for my client to select which modules to release and to provide versions for them ... hopefully this way releasing a new version is just a number of clicks and providing version numbers.
What do you think about this solution? Any Kittens in danger this time? ;-)
Chris
________________________________________
Von: Stephen Connolly [stephen.alan.connolly@gmail.com]
Gesendet: Dienstag, 16. Oktober 2012 11:06
An: Maven Users List
Cc: rwheeler@artifact-software.com
Betreff: Re: Independent module release strategies
On 16 October 2012 07:45, christofer.dutz@c-ware.de <
christofer.dutz@c-ware.de> wrote:
> As described in my other response. Simply keeping the versions in sync is
> not an option for us due to the donwtime this would mean for our clients
> and the load this would generate on the central servers.
>
> Well currently the approach to release a new version was to have all
> modules defined in the master pom modules-section as well as a
> dependencyManagement-section that defines the versions of all the modules.
>
> Now if a new build was to be made that updates only some of the modules,
> the other modules (the ones that should stay the same) were commented out
> of the master poms modules-section and then the releaseplugin was used to
> release the desired artifacts. After the release was finished the versions
> hat do be manually updated.
>
> This process really sucked and caused a lot of problems.
>
> Now my approach was not to use the release plugin at all and to define all
> of the versions used throughout the entire project in the master pom using
> properties. So all I had to do was to increase the versions in the release
> profile to the versions I want and commit that. Now in jenkins I was able
> to define some jobs to run "mvn deploy" for individual projects with turned
> on "release" profile.
>
>
well first off, in my experience, the use of profiles to modify the
dependencies is bad karma. many kittens will die if you follow that use of
profiles. no matter how "clean" you think it is, with the current versions
of Maven and their current behaviour, attempts to follow this path will
result in many dead kittens underfoot.
> To me this seems a lot cleaner than all other approaches, but as I don't
> want to kill too many kittens (As stephen on the dev-list called it). So
> I'm open for other suggestions or explanationy WHY this is bad.
>
> Stephen claimed that if I re-define my properties in child modules, I
> would have really big trouble, but we are developing the entire project and
> this is a thing we could fefinitely rule out because it should be really
> easy to enforce such a constraint ("Versions are defines solely in the
> master pom"). And as I mentioned, to me it looks more like a highly
> restricted feature than a bug in maven that I was able to use a variable in
> the version.
>
>
Not quite, I fear it is loosing something in translation for you.
In an ideal world, before deploying the pom into the local cache (i.e.
install:install) or remote repository (i.e. deploy:deploy) Maven would
compute a resolved effective pom. Such a pom would strip out a lot of the
stuff that is in a pom at present, e.g. it would probably only consist of
/project/groupId
/project/artifactId
/project/version
/project/packaging
/project/dependencies/*
and maybe
/project/build/extensions/*
(but it gets tricky deciding exactly how much to prune out)
Such a pom would be capturing the dependency tree of the built artifact
after inheritance from the parent, and any active profiles, etc had been
applied. [Let's ignore the problem of “magic” profiles that alter the
dependencies when building on JDK1.5 vs JDK1.6 vs JDK1.7 vs JDK1.8]
*IF* we did something like that *THEN* it could be valid to try what you
want to do (Note: I say ‘could be’ not ‘would be’)
This is because anyone depending on the artifact would have a consistent
classpath.
Now for <packaging>pom</packaging> projects we would have to deploy the raw
pom to the repository so that inheritance would continue to work
[thankfully you can only inherit from projects with packaging=pom]. Because
the poms with packaging <> pom would be deploying flattened poms (where
they no-longer inherit from their parent) the fact that the parents are
deployed as raw unflattened poms should/might not be an issue.
That is the ideal world... we are not there... and the picture I paint
above has some holes that need ironing out before we can get there (one
hole for example is GPG signing of poms)
Here is the real world.
I am going to show an issue that you may not have yourself, but which
illustrates the kind/type of problems your approach can result in.
[ Benson I shall now pick on you ;-) ]
Consider the following pom
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-parent</artifactId>
<version>2.7.0</version>
More specifically I would like to draw your attention to the profiles
section
<profiles>
<profile>
<id>jdk17</id>
<activation>
<jdk>1.7</jdk>
</activation>
<properties>
<cxf.jaxb.version>${cxf.jaxb22.version}</cxf.jaxb.version>
<cxf.jaxb.impl.version>${cxf.jaxb22.impl.version}</
cxf.jaxb.impl.version>
<cxf.jaxb.xjc.version>${cxf.jaxb22.impl.version}</
cxf.jaxb.xjc.version>
<jdk.version>1.6</jdk.version>
</properties>
What, you might ask is so wrong with that?
Well what is wrong is that profiles are evaluated at build time.
So at present, if I depend on CXF for building my web application, and I
happen to build *my web application* which only depends on CXF on JDK 1.6
then the JDK 1.7 profile will not be activated because *when maven
evaluates the pom in the repository* it is doing the evaluation with a JDK
1.6 so the profile does not get activated, and thus the web application
will have the wrong jars for running on JDK 1.7. Similar effects can happen
the other way, i.e. if I build my web application with JDK 1.7 but the web
application is supposed to be able to run on JDK 1.6.
[Now Benson is aware of the issues and probably has taken as much care as
he can for CXF, and there is hopefully some trade-off that he could provide
justification for if he wants to]
This is a very real problem, we were badly burned at my previous employers
where some “helpful” person had included a JDK activated profile to correct
for differences between JDK 1.5 and JDK 1.6 which resulted in a broken
product being shipped to customers (who at the time were allowed to use JDK
1.5 or JDK 1.6, and our QA had forgotten to do both tests)
It's not just JDK activation that causes issues!
Think of file based activation... that is evaluated based on the specified
file existing or not... but the file path is relative to the pom.xml... so
at build time you get the profile activated... but as soon as you are
resolving the pom from the repository the profile is no longer activated
(because the file is not in the local repository cache)
this can give great fun where you have a situation where
$ mvn install
works
but then
$ mvn install -f child/pom.xml
does not work, because when we run the full reactor the file activation is
evaluated (because the pom is resolved from the reactor) but when we invoke
the child module directly, the pom is resolved from the repository and so
the file activation is evaluated the other way.
So when is profiles that change the dependencies are evil?
Well there is *one and only one* use case where using profile activation to
manipulate the dependencies is not evil... “<scope>test</scope>”
It is fine to use profiles activated based on the JDK or a file, or system
properties, or the CLI to modify the test classpath... because the test
classpath is not transitive.
The “Maven Way” is to have a build where the artifacts produced are
completely independent of the system on which they were built. That is not
to say that you are killing kittens when you deviate from the Maven way...
more that you are subjecting them to at least mild torture... though move
far enough off the Maven way and there will be dead kittens alright.
A similar evil is using profile activation to change the properties that
get filtered into resources in src/main/resources... (it's ok to change the
properties that get filtered into resources in src/test/resources
though)... but that can be ok in some circumstances (for example baking in
the actual JDK & OS that the product was built with for display in
information/about boxes, etc)
So back to your use case.
When the poms with your “magic” properties are deployed to the repository,
they are missing the information about what profiles were activated via the
CLI and also what system properties were passed via the CLI. So when you
use them as a dependency in a later build the classpath will not be the
same as the one they were built with... much “hilarity” will ensue as you
try to figure out where it all went wrong with your managers breathing down
your neck and the rest of the development team taunting you with the “he
was the one who wanted to move to Maven, we had a nice ANT build that was
working and ‘if it ain't broke, don't fix it’ as they say”
But there is some good news... the solution is to come at the problem from
a different direction.
The first thing I would say is that, 99% of the time, you should have the
Maven Release Plugin using autoVersionSubModules=true. I will go one step
further and say that when you are releasing, everything from the release
root down should have the *exact same version number*
If you need to have different version numbers => they should be separate
release roots and get released independently.
It is a nice ideal to think that you just go
$ mvn release:prepare release:perform -B
from the root of your massive project and everything works... but when the
version numbers of the child projects are different... how will you find
the tagged release of grandchild:1.4.5? oh well that inherits from
child:2.5.3 which inherits from parent:25 => the tag is /tags/parent-25...
hmmm... yes that is obvious
When version numbers are different => a different release lifecycle => a
different release root => release separately.
At my previous employers I wrote the versions-maven-plugin to help managing
dependencies with such a version tree...
so the release process became something like
svn stat
if changes
then
abort and complain need for changes to commit
fi
mvn versions:update-properties
svn stat
if changes
then
svn commit -m "update dependencies"
fi
if not svn log -l 1 | grep "[maven-release-plugin]"
then
mvn release:prepare release:perform -B
else
echo "no changes, nothing to do"
fi
we actually had a Maven plugin that did the above using forked processes
and the SCM shared utils (unfortunately one that was not developed “while
travelling between home and the office” and therefore was developed while I
was “engaged in the employ of my employers” and hence not one I could open
source [got to love the Health and Safety exception in the Irish statute
book... my contract stated that anything I developed while “engaged in the
employ of ___” belonged to them, and the Irish statute book only
mentions “engaged in the employ” in H&S where it states that you are
*specifically not engaged in the employ of your employers while commuting
between home and the office*])
Since I don't have that problem with my current employers, where we are a
cloud based PaaS and basically, barring exceptions, always ensure that the
head release is either failing to build or safe to push to production
(a.k.a. continuous deployment - because if it is failing to build then we
fix that damn fast)
And since I do not have any free time at present, I am not invested in
re-implementing my magic-release plugin... though it would be super handy
for lots of people... in part because it identified the dependency chain
and evaluated the release roots in the order in which they were required to
be evaluated, thereby ensuring that only the minimum required release set
was released.
Side-note: of course the reason we didn't do the big bang release
everything was because some of our customers were on dial-up modems...
never mind that some Architect decided that we were going to switch to
delivering the software as VM images so that even a 1 byte change ment the
customer re-downloading 15GB of VM images :rolleyes:
Chris
>
>
>
> ________________________________________
> Von: Ron Wheeler [rwheeler@artifact-software.com]
> Gesendet: Dienstag, 16. Oktober 2012 00:21
> An: users@maven.apache.org
> Betreff: Re: Independent module release strategies
>
> Our project was about 50% larger in terms of modules and we did what you
> are suggesting and got rid of the idea of increasing versions on
> artifacts that did not change.
>
> I am not sure why this is causing you more problems rather than less.
>
> We had a master spreadsheet listing all of our modules and their versions.
> Every time we issued a new release, we went through the list at looked
> at what was going to change and what was going to carry through "as is"
> with the version number that it had.
>
> We also looked at third party libraries that we wanted to upgrade at the
> same time.
>
> We fixed up each pom to update the versions of anything that was going
> to change to a x.x-SNAPSHOT and moved on.
> It took about an hour to do the whole job since our system was service
> oriented so there were not a lot of dependencies.
>
> Once a module was tested and released we updated the dependencies to the
> released module.
> We still ended up with a lot of releases at the very end of the upgrade
> process but that is partly human nature since deleting a release is a
> bad thing even if it is one of yours and could only be done by me so it
> got a lot of visibility if someone made a mistake.
> It is hard to get everyone confident that their module's specification
> will not change due to someone else making a mistake in their design
> which only gets detected late in the process. They solved this by
> staying at the SNAPSHOT after they had it fully tested and ready for
> release, if someone who depended on it was not yet done.
> It was not a big problem and I never took any steps to fix it.
>
> We aggregated a lot of library-like dependencies into larger packages
> that were "provided". This gave a dependency on our "utilities" package
> that actually was an aggregation of several projects so each war project
> did not have dozens of dependencies on modules that were shared by most
> modules.
> We did this with third party software as well so a lot of really useful
> Apache libraries were aggregated into 1 jar that all projects depended on.
>
> This reduced the number of dependencies in the POM files and made them a
> lot easier to maintain.
> By using a lot of "Provided" jars, we really sped up the builds and
> reduced the size of the war files from megabytes to kilobytes since they
> only contained the code that was in the source files rather that a few
> Kb of code output and megabytes of libraries.
>
> I am not sure that this is the best way to tackle the problem but it
> eliminated the work that we were doing when we changed the version on
> everything that made up the application.
>
> It also got us thinking about our own packages in the same way that we
> looked at Apache libraries.
>
> There was an incentive to think about interfaces and SOA in a more
> considered way.
>
> I hope that this helps.
> You are on the right track and your project is still pretty small at 50
> modules.
>
> Ron
>
> On 15/10/2012 5:32 PM, christofer.dutz@c-ware.de wrote:
> >
> > Hi,
> >
> > I am currently working on finetuning the workflows on a large
> > application that was migrated from an Ant based build to one based
> > upon Maven.
> >
> > The build itself is running smoothly but, what I'm currently working
> > on is getting the release workflow optimized.
> >
> > The project consists of about 50 Maven artifacts. A lot of people are
> > using this project all over the world. The client is distributed by
> > some web-start similar solution.
> >
> > The problem is whenever a bugfix-release is done, we don't want to
> > release all modules in a new version because then all of these would
> > have to be downloaded by the clients.
> >
> > So we have a project with a lot of modules and a parent pom that
> > configures the plugins.
> >
> > Using the regular maven-release-plugin involves a lot of manual
> > adjusting of version numbers and I would like to eliminate this.
> >
> > That's why I setup the master pom to have two profiles "develop"
> > (active by default) and "release" activated during a release. In both
> > profiles a lot of properties are configured to be used for setting the
> > artifact versions.
> >
> > No comes the part where I was told on the dev-list that I was tempted
> > by the dark side of the force ;-)
> >
> > In my master pom, I defined one major dependencyManagement section
> > fixing the version of each artifact to the versions in the properties.
> > Ok ... so this is normal and this is not "dark side magic". But in
> > order to have the parent version automatically configured the right
> > way I wanted to have the version of the parent link configured by
> > these properties too. Ok so maven doesn't allow this. But it seems
> > that this is not entirely true:
> >
> > If I configre the version of the artifact I want to use as parent with
> > the same variable as I am using in the parent definition of the child
> > module. Maven seems to work fine with that. The only thing that I was
> > not quite satisfied with, was that the install plugin installed the
> > raw poms into my local repo. The directory it was installed to
> > contained the correct version so the resolution must have worked.
> > That's why I thought this was a bug in the deploy plugin and that's
> > why I filed an issue (which was immediately closed because I was doing
> > bad magic) http://jira.codehaus.org/browse/MNG-5358
> >
> > I attached an example project containing an example configuration
> > demonstrating what I was doing.
> >
> > What I find particularly strange is that Maven claims not to resolve
> > properties in project.version and project.parent.version and in 90% of
> > the cases this is true:
> >
> > Let me illustrate thsi a little. Assuming I have only two projects ...
> > one master and one module.
> >
> > If I define two properties in my master pom: "my.cool.master.version"
> > and "my.cool.alternate.master.version" and set both to the same value
> > of "1.2-SNAPSHOT".
> >
> > In szenario 1: I hard code the version of the master to "1.2-SNAPSHOT"
> > but use the property to reference the parent from the moule ... when
> > running a build, maven tries to download
> >
> "de/mycompany/test/${my.cool.master.version}/mycoolmaster-${my.cool.master.version}.pom"
> > --> Failure, because the property is not resolved
> >
> > In szenario 2: I use the same variable for defining the masters
> > version. This time the maven build runs fine and the parent version is
> > correctly resolved.
> >
> > In szenario 3: I use the first property to define the version of the
> > master and the second one for referncing the parent from the module
> > ... when running a build, maven tries to download
> >
> "de/mycompany/test/${my.cool.alternate.master.version}/mycoolmaster-${my.cool.alternate.master.version}.pom"
> > --> Failure, because the property is not resolved
> >
> > So to me it looks as if there was some sort of intention behind
> > everything and not a bug in the system as I was told on the dev-list.
> > To me it looks like one teeniewienie loophole allowing properies in
> > versions while closing the usage range so much that possible harm is
> > reduced to it's absolute minimum. So it seems that my usecase seems to
> > be the onlly one allowed. After all ... this is a problem users are
> > begging for maven to provide a solution since maven 2.0 (When looking
> > at the forums).
> >
> > Ok ... and now to finish the loop back to my topic:
> >
> > If I am doing bad sourcery ... how would I setup one maven build to
> > allow simple releases of individual modules with individual versions?
> >
> > Chris
> >
> > [ C h r i s t o f e r D u t z ]
> >
> > C-Ware IT-Service
> >
> > Inhaber
> >
> > Dipl. Inf. Christofer Dutz
> >
> > Karlstraße. 104, 64285 Darmstadt
> >
> > Beschreibung: 788335
> > <http://www.benchpark.com/788335/kundenzufriedenheit.htm>
> >
> > IT- und Systemhäuser <http://www.benchpark.com/it_und_systemhaeuser.htm>
> >
> > fon: 0 61 51 / 27315 - 61
> >
> > fax: 0 61 51 / 27315 - 64
> >
> > mobil: 0171 / 7 444 2 33
> >
> > email: christofer.dutz@c-ware.de <ma...@c-ware.de>
> >
> > http://www.c-ware.de <http://www.c-ware.de/>
> >
> > UStId-Nr. DE195700962
> >
>
>
> --
> Ron Wheeler
> President
> Artifact Software Inc
> email: rwheeler@artifact-software.com
> skype: ronaldmwheeler
> phone: 866-970-2435, ext 102
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Independent module release strategies
Posted by Stephen Connolly <st...@gmail.com>.
On 16 October 2012 07:45, christofer.dutz@c-ware.de <
christofer.dutz@c-ware.de> wrote:
> As described in my other response. Simply keeping the versions in sync is
> not an option for us due to the donwtime this would mean for our clients
> and the load this would generate on the central servers.
>
> Well currently the approach to release a new version was to have all
> modules defined in the master pom modules-section as well as a
> dependencyManagement-section that defines the versions of all the modules.
>
> Now if a new build was to be made that updates only some of the modules,
> the other modules (the ones that should stay the same) were commented out
> of the master poms modules-section and then the releaseplugin was used to
> release the desired artifacts. After the release was finished the versions
> hat do be manually updated.
>
> This process really sucked and caused a lot of problems.
>
> Now my approach was not to use the release plugin at all and to define all
> of the versions used throughout the entire project in the master pom using
> properties. So all I had to do was to increase the versions in the release
> profile to the versions I want and commit that. Now in jenkins I was able
> to define some jobs to run "mvn deploy" for individual projects with turned
> on "release" profile.
>
>
well first off, in my experience, the use of profiles to modify the
dependencies is bad karma. many kittens will die if you follow that use of
profiles. no matter how "clean" you think it is, with the current versions
of Maven and their current behaviour, attempts to follow this path will
result in many dead kittens underfoot.
> To me this seems a lot cleaner than all other approaches, but as I don't
> want to kill too many kittens (As stephen on the dev-list called it). So
> I'm open for other suggestions or explanationy WHY this is bad.
>
> Stephen claimed that if I re-define my properties in child modules, I
> would have really big trouble, but we are developing the entire project and
> this is a thing we could fefinitely rule out because it should be really
> easy to enforce such a constraint ("Versions are defines solely in the
> master pom"). And as I mentioned, to me it looks more like a highly
> restricted feature than a bug in maven that I was able to use a variable in
> the version.
>
>
Not quite, I fear it is loosing something in translation for you.
In an ideal world, before deploying the pom into the local cache (i.e.
install:install) or remote repository (i.e. deploy:deploy) Maven would
compute a resolved effective pom. Such a pom would strip out a lot of the
stuff that is in a pom at present, e.g. it would probably only consist of
/project/groupId
/project/artifactId
/project/version
/project/packaging
/project/dependencies/*
and maybe
/project/build/extensions/*
(but it gets tricky deciding exactly how much to prune out)
Such a pom would be capturing the dependency tree of the built artifact
after inheritance from the parent, and any active profiles, etc had been
applied. [Let's ignore the problem of “magic” profiles that alter the
dependencies when building on JDK1.5 vs JDK1.6 vs JDK1.7 vs JDK1.8]
*IF* we did something like that *THEN* it could be valid to try what you
want to do (Note: I say ‘could be’ not ‘would be’)
This is because anyone depending on the artifact would have a consistent
classpath.
Now for <packaging>pom</packaging> projects we would have to deploy the raw
pom to the repository so that inheritance would continue to work
[thankfully you can only inherit from projects with packaging=pom]. Because
the poms with packaging <> pom would be deploying flattened poms (where
they no-longer inherit from their parent) the fact that the parents are
deployed as raw unflattened poms should/might not be an issue.
That is the ideal world... we are not there... and the picture I paint
above has some holes that need ironing out before we can get there (one
hole for example is GPG signing of poms)
Here is the real world.
I am going to show an issue that you may not have yourself, but which
illustrates the kind/type of problems your approach can result in.
[ Benson I shall now pick on you ;-) ]
Consider the following pom
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-parent</artifactId>
<version>2.7.0</version>
More specifically I would like to draw your attention to the profiles
section
<profiles>
<profile>
<id>jdk17</id>
<activation>
<jdk>1.7</jdk>
</activation>
<properties>
<cxf.jaxb.version>${cxf.jaxb22.version}</cxf.jaxb.version>
<cxf.jaxb.impl.version>${cxf.jaxb22.impl.version}</
cxf.jaxb.impl.version>
<cxf.jaxb.xjc.version>${cxf.jaxb22.impl.version}</
cxf.jaxb.xjc.version>
<jdk.version>1.6</jdk.version>
</properties>
What, you might ask is so wrong with that?
Well what is wrong is that profiles are evaluated at build time.
So at present, if I depend on CXF for building my web application, and I
happen to build *my web application* which only depends on CXF on JDK 1.6
then the JDK 1.7 profile will not be activated because *when maven
evaluates the pom in the repository* it is doing the evaluation with a JDK
1.6 so the profile does not get activated, and thus the web application
will have the wrong jars for running on JDK 1.7. Similar effects can happen
the other way, i.e. if I build my web application with JDK 1.7 but the web
application is supposed to be able to run on JDK 1.6.
[Now Benson is aware of the issues and probably has taken as much care as
he can for CXF, and there is hopefully some trade-off that he could provide
justification for if he wants to]
This is a very real problem, we were badly burned at my previous employers
where some “helpful” person had included a JDK activated profile to correct
for differences between JDK 1.5 and JDK 1.6 which resulted in a broken
product being shipped to customers (who at the time were allowed to use JDK
1.5 or JDK 1.6, and our QA had forgotten to do both tests)
It's not just JDK activation that causes issues!
Think of file based activation... that is evaluated based on the specified
file existing or not... but the file path is relative to the pom.xml... so
at build time you get the profile activated... but as soon as you are
resolving the pom from the repository the profile is no longer activated
(because the file is not in the local repository cache)
this can give great fun where you have a situation where
$ mvn install
works
but then
$ mvn install -f child/pom.xml
does not work, because when we run the full reactor the file activation is
evaluated (because the pom is resolved from the reactor) but when we invoke
the child module directly, the pom is resolved from the repository and so
the file activation is evaluated the other way.
So when is profiles that change the dependencies are evil?
Well there is *one and only one* use case where using profile activation to
manipulate the dependencies is not evil... “<scope>test</scope>”
It is fine to use profiles activated based on the JDK or a file, or system
properties, or the CLI to modify the test classpath... because the test
classpath is not transitive.
The “Maven Way” is to have a build where the artifacts produced are
completely independent of the system on which they were built. That is not
to say that you are killing kittens when you deviate from the Maven way...
more that you are subjecting them to at least mild torture... though move
far enough off the Maven way and there will be dead kittens alright.
A similar evil is using profile activation to change the properties that
get filtered into resources in src/main/resources... (it's ok to change the
properties that get filtered into resources in src/test/resources
though)... but that can be ok in some circumstances (for example baking in
the actual JDK & OS that the product was built with for display in
information/about boxes, etc)
So back to your use case.
When the poms with your “magic” properties are deployed to the repository,
they are missing the information about what profiles were activated via the
CLI and also what system properties were passed via the CLI. So when you
use them as a dependency in a later build the classpath will not be the
same as the one they were built with... much “hilarity” will ensue as you
try to figure out where it all went wrong with your managers breathing down
your neck and the rest of the development team taunting you with the “he
was the one who wanted to move to Maven, we had a nice ANT build that was
working and ‘if it ain't broke, don't fix it’ as they say”
But there is some good news... the solution is to come at the problem from
a different direction.
The first thing I would say is that, 99% of the time, you should have the
Maven Release Plugin using autoVersionSubModules=true. I will go one step
further and say that when you are releasing, everything from the release
root down should have the *exact same version number*
If you need to have different version numbers => they should be separate
release roots and get released independently.
It is a nice ideal to think that you just go
$ mvn release:prepare release:perform -B
from the root of your massive project and everything works... but when the
version numbers of the child projects are different... how will you find
the tagged release of grandchild:1.4.5? oh well that inherits from
child:2.5.3 which inherits from parent:25 => the tag is /tags/parent-25...
hmmm... yes that is obvious
When version numbers are different => a different release lifecycle => a
different release root => release separately.
At my previous employers I wrote the versions-maven-plugin to help managing
dependencies with such a version tree...
so the release process became something like
svn stat
if changes
then
abort and complain need for changes to commit
fi
mvn versions:update-properties
svn stat
if changes
then
svn commit -m "update dependencies"
fi
if not svn log -l 1 | grep "[maven-release-plugin]"
then
mvn release:prepare release:perform -B
else
echo "no changes, nothing to do"
fi
we actually had a Maven plugin that did the above using forked processes
and the SCM shared utils (unfortunately one that was not developed “while
travelling between home and the office” and therefore was developed while I
was “engaged in the employ of my employers” and hence not one I could open
source [got to love the Health and Safety exception in the Irish statute
book... my contract stated that anything I developed while “engaged in the
employ of ___” belonged to them, and the Irish statute book only
mentions “engaged in the employ” in H&S where it states that you are
*specifically not engaged in the employ of your employers while commuting
between home and the office*])
Since I don't have that problem with my current employers, where we are a
cloud based PaaS and basically, barring exceptions, always ensure that the
head release is either failing to build or safe to push to production
(a.k.a. continuous deployment - because if it is failing to build then we
fix that damn fast)
And since I do not have any free time at present, I am not invested in
re-implementing my magic-release plugin... though it would be super handy
for lots of people... in part because it identified the dependency chain
and evaluated the release roots in the order in which they were required to
be evaluated, thereby ensuring that only the minimum required release set
was released.
Side-note: of course the reason we didn't do the big bang release
everything was because some of our customers were on dial-up modems...
never mind that some Architect decided that we were going to switch to
delivering the software as VM images so that even a 1 byte change ment the
customer re-downloading 15GB of VM images :rolleyes:
Chris
>
>
>
> ________________________________________
> Von: Ron Wheeler [rwheeler@artifact-software.com]
> Gesendet: Dienstag, 16. Oktober 2012 00:21
> An: users@maven.apache.org
> Betreff: Re: Independent module release strategies
>
> Our project was about 50% larger in terms of modules and we did what you
> are suggesting and got rid of the idea of increasing versions on
> artifacts that did not change.
>
> I am not sure why this is causing you more problems rather than less.
>
> We had a master spreadsheet listing all of our modules and their versions.
> Every time we issued a new release, we went through the list at looked
> at what was going to change and what was going to carry through "as is"
> with the version number that it had.
>
> We also looked at third party libraries that we wanted to upgrade at the
> same time.
>
> We fixed up each pom to update the versions of anything that was going
> to change to a x.x-SNAPSHOT and moved on.
> It took about an hour to do the whole job since our system was service
> oriented so there were not a lot of dependencies.
>
> Once a module was tested and released we updated the dependencies to the
> released module.
> We still ended up with a lot of releases at the very end of the upgrade
> process but that is partly human nature since deleting a release is a
> bad thing even if it is one of yours and could only be done by me so it
> got a lot of visibility if someone made a mistake.
> It is hard to get everyone confident that their module's specification
> will not change due to someone else making a mistake in their design
> which only gets detected late in the process. They solved this by
> staying at the SNAPSHOT after they had it fully tested and ready for
> release, if someone who depended on it was not yet done.
> It was not a big problem and I never took any steps to fix it.
>
> We aggregated a lot of library-like dependencies into larger packages
> that were "provided". This gave a dependency on our "utilities" package
> that actually was an aggregation of several projects so each war project
> did not have dozens of dependencies on modules that were shared by most
> modules.
> We did this with third party software as well so a lot of really useful
> Apache libraries were aggregated into 1 jar that all projects depended on.
>
> This reduced the number of dependencies in the POM files and made them a
> lot easier to maintain.
> By using a lot of "Provided" jars, we really sped up the builds and
> reduced the size of the war files from megabytes to kilobytes since they
> only contained the code that was in the source files rather that a few
> Kb of code output and megabytes of libraries.
>
> I am not sure that this is the best way to tackle the problem but it
> eliminated the work that we were doing when we changed the version on
> everything that made up the application.
>
> It also got us thinking about our own packages in the same way that we
> looked at Apache libraries.
>
> There was an incentive to think about interfaces and SOA in a more
> considered way.
>
> I hope that this helps.
> You are on the right track and your project is still pretty small at 50
> modules.
>
> Ron
>
> On 15/10/2012 5:32 PM, christofer.dutz@c-ware.de wrote:
> >
> > Hi,
> >
> > I am currently working on finetuning the workflows on a large
> > application that was migrated from an Ant based build to one based
> > upon Maven.
> >
> > The build itself is running smoothly but, what I'm currently working
> > on is getting the release workflow optimized.
> >
> > The project consists of about 50 Maven artifacts. A lot of people are
> > using this project all over the world. The client is distributed by
> > some web-start similar solution.
> >
> > The problem is whenever a bugfix-release is done, we don't want to
> > release all modules in a new version because then all of these would
> > have to be downloaded by the clients.
> >
> > So we have a project with a lot of modules and a parent pom that
> > configures the plugins.
> >
> > Using the regular maven-release-plugin involves a lot of manual
> > adjusting of version numbers and I would like to eliminate this.
> >
> > That's why I setup the master pom to have two profiles "develop"
> > (active by default) and "release" activated during a release. In both
> > profiles a lot of properties are configured to be used for setting the
> > artifact versions.
> >
> > No comes the part where I was told on the dev-list that I was tempted
> > by the dark side of the force ;-)
> >
> > In my master pom, I defined one major dependencyManagement section
> > fixing the version of each artifact to the versions in the properties.
> > Ok ... so this is normal and this is not "dark side magic". But in
> > order to have the parent version automatically configured the right
> > way I wanted to have the version of the parent link configured by
> > these properties too. Ok so maven doesn't allow this. But it seems
> > that this is not entirely true:
> >
> > If I configre the version of the artifact I want to use as parent with
> > the same variable as I am using in the parent definition of the child
> > module. Maven seems to work fine with that. The only thing that I was
> > not quite satisfied with, was that the install plugin installed the
> > raw poms into my local repo. The directory it was installed to
> > contained the correct version so the resolution must have worked.
> > That's why I thought this was a bug in the deploy plugin and that's
> > why I filed an issue (which was immediately closed because I was doing
> > bad magic) http://jira.codehaus.org/browse/MNG-5358
> >
> > I attached an example project containing an example configuration
> > demonstrating what I was doing.
> >
> > What I find particularly strange is that Maven claims not to resolve
> > properties in project.version and project.parent.version and in 90% of
> > the cases this is true:
> >
> > Let me illustrate thsi a little. Assuming I have only two projects ...
> > one master and one module.
> >
> > If I define two properties in my master pom: "my.cool.master.version"
> > and "my.cool.alternate.master.version" and set both to the same value
> > of "1.2-SNAPSHOT".
> >
> > In szenario 1: I hard code the version of the master to "1.2-SNAPSHOT"
> > but use the property to reference the parent from the moule ... when
> > running a build, maven tries to download
> >
> "de/mycompany/test/${my.cool.master.version}/mycoolmaster-${my.cool.master.version}.pom"
> > --> Failure, because the property is not resolved
> >
> > In szenario 2: I use the same variable for defining the masters
> > version. This time the maven build runs fine and the parent version is
> > correctly resolved.
> >
> > In szenario 3: I use the first property to define the version of the
> > master and the second one for referncing the parent from the module
> > ... when running a build, maven tries to download
> >
> "de/mycompany/test/${my.cool.alternate.master.version}/mycoolmaster-${my.cool.alternate.master.version}.pom"
> > --> Failure, because the property is not resolved
> >
> > So to me it looks as if there was some sort of intention behind
> > everything and not a bug in the system as I was told on the dev-list.
> > To me it looks like one teeniewienie loophole allowing properies in
> > versions while closing the usage range so much that possible harm is
> > reduced to it's absolute minimum. So it seems that my usecase seems to
> > be the onlly one allowed. After all ... this is a problem users are
> > begging for maven to provide a solution since maven 2.0 (When looking
> > at the forums).
> >
> > Ok ... and now to finish the loop back to my topic:
> >
> > If I am doing bad sourcery ... how would I setup one maven build to
> > allow simple releases of individual modules with individual versions?
> >
> > Chris
> >
> > [ C h r i s t o f e r D u t z ]
> >
> > C-Ware IT-Service
> >
> > Inhaber
> >
> > Dipl. Inf. Christofer Dutz
> >
> > Karlstraße. 104, 64285 Darmstadt
> >
> > Beschreibung: 788335
> > <http://www.benchpark.com/788335/kundenzufriedenheit.htm>
> >
> > IT- und Systemhäuser <http://www.benchpark.com/it_und_systemhaeuser.htm>
> >
> > fon: 0 61 51 / 27315 - 61
> >
> > fax: 0 61 51 / 27315 - 64
> >
> > mobil: 0171 / 7 444 2 33
> >
> > email: christofer.dutz@c-ware.de <ma...@c-ware.de>
> >
> > http://www.c-ware.de <http://www.c-ware.de/>
> >
> > UStId-Nr. DE195700962
> >
>
>
> --
> Ron Wheeler
> President
> Artifact Software Inc
> email: rwheeler@artifact-software.com
> skype: ronaldmwheeler
> phone: 866-970-2435, ext 102
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
>
AW: Independent module release strategies
Posted by "christofer.dutz@c-ware.de" <ch...@c-ware.de>.
As described in my other response. Simply keeping the versions in sync is not an option for us due to the donwtime this would mean for our clients and the load this would generate on the central servers.
Well currently the approach to release a new version was to have all modules defined in the master pom modules-section as well as a dependencyManagement-section that defines the versions of all the modules.
Now if a new build was to be made that updates only some of the modules, the other modules (the ones that should stay the same) were commented out of the master poms modules-section and then the releaseplugin was used to release the desired artifacts. After the release was finished the versions hat do be manually updated.
This process really sucked and caused a lot of problems.
Now my approach was not to use the release plugin at all and to define all of the versions used throughout the entire project in the master pom using properties. So all I had to do was to increase the versions in the release profile to the versions I want and commit that. Now in jenkins I was able to define some jobs to run "mvn deploy" for individual projects with turned on "release" profile.
To me this seems a lot cleaner than all other approaches, but as I don't want to kill too many kittens (As stephen on the dev-list called it). So I'm open for other suggestions or explanationy WHY this is bad.
Stephen claimed that if I re-define my properties in child modules, I would have really big trouble, but we are developing the entire project and this is a thing we could fefinitely rule out because it should be really easy to enforce such a constraint ("Versions are defines solely in the master pom"). And as I mentioned, to me it looks more like a highly restricted feature than a bug in maven that I was able to use a variable in the version.
Chris
________________________________________
Von: Ron Wheeler [rwheeler@artifact-software.com]
Gesendet: Dienstag, 16. Oktober 2012 00:21
An: users@maven.apache.org
Betreff: Re: Independent module release strategies
Our project was about 50% larger in terms of modules and we did what you
are suggesting and got rid of the idea of increasing versions on
artifacts that did not change.
I am not sure why this is causing you more problems rather than less.
We had a master spreadsheet listing all of our modules and their versions.
Every time we issued a new release, we went through the list at looked
at what was going to change and what was going to carry through "as is"
with the version number that it had.
We also looked at third party libraries that we wanted to upgrade at the
same time.
We fixed up each pom to update the versions of anything that was going
to change to a x.x-SNAPSHOT and moved on.
It took about an hour to do the whole job since our system was service
oriented so there were not a lot of dependencies.
Once a module was tested and released we updated the dependencies to the
released module.
We still ended up with a lot of releases at the very end of the upgrade
process but that is partly human nature since deleting a release is a
bad thing even if it is one of yours and could only be done by me so it
got a lot of visibility if someone made a mistake.
It is hard to get everyone confident that their module's specification
will not change due to someone else making a mistake in their design
which only gets detected late in the process. They solved this by
staying at the SNAPSHOT after they had it fully tested and ready for
release, if someone who depended on it was not yet done.
It was not a big problem and I never took any steps to fix it.
We aggregated a lot of library-like dependencies into larger packages
that were "provided". This gave a dependency on our "utilities" package
that actually was an aggregation of several projects so each war project
did not have dozens of dependencies on modules that were shared by most
modules.
We did this with third party software as well so a lot of really useful
Apache libraries were aggregated into 1 jar that all projects depended on.
This reduced the number of dependencies in the POM files and made them a
lot easier to maintain.
By using a lot of "Provided" jars, we really sped up the builds and
reduced the size of the war files from megabytes to kilobytes since they
only contained the code that was in the source files rather that a few
Kb of code output and megabytes of libraries.
I am not sure that this is the best way to tackle the problem but it
eliminated the work that we were doing when we changed the version on
everything that made up the application.
It also got us thinking about our own packages in the same way that we
looked at Apache libraries.
There was an incentive to think about interfaces and SOA in a more
considered way.
I hope that this helps.
You are on the right track and your project is still pretty small at 50
modules.
Ron
On 15/10/2012 5:32 PM, christofer.dutz@c-ware.de wrote:
>
> Hi,
>
> I am currently working on finetuning the workflows on a large
> application that was migrated from an Ant based build to one based
> upon Maven.
>
> The build itself is running smoothly but, what I'm currently working
> on is getting the release workflow optimized.
>
> The project consists of about 50 Maven artifacts. A lot of people are
> using this project all over the world. The client is distributed by
> some web-start similar solution.
>
> The problem is whenever a bugfix-release is done, we don't want to
> release all modules in a new version because then all of these would
> have to be downloaded by the clients.
>
> So we have a project with a lot of modules and a parent pom that
> configures the plugins.
>
> Using the regular maven-release-plugin involves a lot of manual
> adjusting of version numbers and I would like to eliminate this.
>
> That's why I setup the master pom to have two profiles "develop"
> (active by default) and "release" activated during a release. In both
> profiles a lot of properties are configured to be used for setting the
> artifact versions.
>
> No comes the part where I was told on the dev-list that I was tempted
> by the dark side of the force ;-)
>
> In my master pom, I defined one major dependencyManagement section
> fixing the version of each artifact to the versions in the properties.
> Ok ... so this is normal and this is not "dark side magic". But in
> order to have the parent version automatically configured the right
> way I wanted to have the version of the parent link configured by
> these properties too. Ok so maven doesn't allow this. But it seems
> that this is not entirely true:
>
> If I configre the version of the artifact I want to use as parent with
> the same variable as I am using in the parent definition of the child
> module. Maven seems to work fine with that. The only thing that I was
> not quite satisfied with, was that the install plugin installed the
> raw poms into my local repo. The directory it was installed to
> contained the correct version so the resolution must have worked.
> That's why I thought this was a bug in the deploy plugin and that's
> why I filed an issue (which was immediately closed because I was doing
> bad magic) http://jira.codehaus.org/browse/MNG-5358
>
> I attached an example project containing an example configuration
> demonstrating what I was doing.
>
> What I find particularly strange is that Maven claims not to resolve
> properties in project.version and project.parent.version and in 90% of
> the cases this is true:
>
> Let me illustrate thsi a little. Assuming I have only two projects ...
> one master and one module.
>
> If I define two properties in my master pom: "my.cool.master.version"
> and "my.cool.alternate.master.version" and set both to the same value
> of "1.2-SNAPSHOT".
>
> In szenario 1: I hard code the version of the master to "1.2-SNAPSHOT"
> but use the property to reference the parent from the moule ... when
> running a build, maven tries to download
> "de/mycompany/test/${my.cool.master.version}/mycoolmaster-${my.cool.master.version}.pom"
> --> Failure, because the property is not resolved
>
> In szenario 2: I use the same variable for defining the masters
> version. This time the maven build runs fine and the parent version is
> correctly resolved.
>
> In szenario 3: I use the first property to define the version of the
> master and the second one for referncing the parent from the module
> ... when running a build, maven tries to download
> "de/mycompany/test/${my.cool.alternate.master.version}/mycoolmaster-${my.cool.alternate.master.version}.pom"
> --> Failure, because the property is not resolved
>
> So to me it looks as if there was some sort of intention behind
> everything and not a bug in the system as I was told on the dev-list.
> To me it looks like one teeniewienie loophole allowing properies in
> versions while closing the usage range so much that possible harm is
> reduced to it's absolute minimum. So it seems that my usecase seems to
> be the onlly one allowed. After all ... this is a problem users are
> begging for maven to provide a solution since maven 2.0 (When looking
> at the forums).
>
> Ok ... and now to finish the loop back to my topic:
>
> If I am doing bad sourcery ... how would I setup one maven build to
> allow simple releases of individual modules with individual versions?
>
> Chris
>
> [ C h r i s t o f e r D u t z ]
>
> C-Ware IT-Service
>
> Inhaber
>
> Dipl. Inf. Christofer Dutz
>
> Karlstraße. 104, 64285 Darmstadt
>
> Beschreibung: 788335
> <http://www.benchpark.com/788335/kundenzufriedenheit.htm>
>
> IT- und Systemhäuser <http://www.benchpark.com/it_und_systemhaeuser.htm>
>
> fon: 0 61 51 / 27315 - 61
>
> fax: 0 61 51 / 27315 - 64
>
> mobil: 0171 / 7 444 2 33
>
> email: christofer.dutz@c-ware.de <ma...@c-ware.de>
>
> http://www.c-ware.de <http://www.c-ware.de/>
>
> UStId-Nr. DE195700962
>
--
Ron Wheeler
President
Artifact Software Inc
email: rwheeler@artifact-software.com
skype: ronaldmwheeler
phone: 866-970-2435, ext 102
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Independent module release strategies
Posted by Ron Wheeler <rw...@artifact-software.com>.
Our project was about 50% larger in terms of modules and we did what you
are suggesting and got rid of the idea of increasing versions on
artifacts that did not change.
I am not sure why this is causing you more problems rather than less.
We had a master spreadsheet listing all of our modules and their versions.
Every time we issued a new release, we went through the list at looked
at what was going to change and what was going to carry through "as is"
with the version number that it had.
We also looked at third party libraries that we wanted to upgrade at the
same time.
We fixed up each pom to update the versions of anything that was going
to change to a x.x-SNAPSHOT and moved on.
It took about an hour to do the whole job since our system was service
oriented so there were not a lot of dependencies.
Once a module was tested and released we updated the dependencies to the
released module.
We still ended up with a lot of releases at the very end of the upgrade
process but that is partly human nature since deleting a release is a
bad thing even if it is one of yours and could only be done by me so it
got a lot of visibility if someone made a mistake.
It is hard to get everyone confident that their module's specification
will not change due to someone else making a mistake in their design
which only gets detected late in the process. They solved this by
staying at the SNAPSHOT after they had it fully tested and ready for
release, if someone who depended on it was not yet done.
It was not a big problem and I never took any steps to fix it.
We aggregated a lot of library-like dependencies into larger packages
that were "provided". This gave a dependency on our "utilities" package
that actually was an aggregation of several projects so each war project
did not have dozens of dependencies on modules that were shared by most
modules.
We did this with third party software as well so a lot of really useful
Apache libraries were aggregated into 1 jar that all projects depended on.
This reduced the number of dependencies in the POM files and made them a
lot easier to maintain.
By using a lot of "Provided" jars, we really sped up the builds and
reduced the size of the war files from megabytes to kilobytes since they
only contained the code that was in the source files rather that a few
Kb of code output and megabytes of libraries.
I am not sure that this is the best way to tackle the problem but it
eliminated the work that we were doing when we changed the version on
everything that made up the application.
It also got us thinking about our own packages in the same way that we
looked at Apache libraries.
There was an incentive to think about interfaces and SOA in a more
considered way.
I hope that this helps.
You are on the right track and your project is still pretty small at 50
modules.
Ron
On 15/10/2012 5:32 PM, christofer.dutz@c-ware.de wrote:
>
> Hi,
>
> I am currently working on finetuning the workflows on a large
> application that was migrated from an Ant based build to one based
> upon Maven.
>
> The build itself is running smoothly but, what I'm currently working
> on is getting the release workflow optimized.
>
> The project consists of about 50 Maven artifacts. A lot of people are
> using this project all over the world. The client is distributed by
> some web-start similar solution.
>
> The problem is whenever a bugfix-release is done, we don't want to
> release all modules in a new version because then all of these would
> have to be downloaded by the clients.
>
> So we have a project with a lot of modules and a parent pom that
> configures the plugins.
>
> Using the regular maven-release-plugin involves a lot of manual
> adjusting of version numbers and I would like to eliminate this.
>
> That's why I setup the master pom to have two profiles "develop"
> (active by default) and "release" activated during a release. In both
> profiles a lot of properties are configured to be used for setting the
> artifact versions.
>
> No comes the part where I was told on the dev-list that I was tempted
> by the dark side of the force ;-)
>
> In my master pom, I defined one major dependencyManagement section
> fixing the version of each artifact to the versions in the properties.
> Ok ... so this is normal and this is not "dark side magic". But in
> order to have the parent version automatically configured the right
> way I wanted to have the version of the parent link configured by
> these properties too. Ok so maven doesn't allow this. But it seems
> that this is not entirely true:
>
> If I configre the version of the artifact I want to use as parent with
> the same variable as I am using in the parent definition of the child
> module. Maven seems to work fine with that. The only thing that I was
> not quite satisfied with, was that the install plugin installed the
> raw poms into my local repo. The directory it was installed to
> contained the correct version so the resolution must have worked.
> That's why I thought this was a bug in the deploy plugin and that's
> why I filed an issue (which was immediately closed because I was doing
> bad magic) http://jira.codehaus.org/browse/MNG-5358
>
> I attached an example project containing an example configuration
> demonstrating what I was doing.
>
> What I find particularly strange is that Maven claims not to resolve
> properties in project.version and project.parent.version and in 90% of
> the cases this is true:
>
> Let me illustrate thsi a little. Assuming I have only two projects ...
> one master and one module.
>
> If I define two properties in my master pom: "my.cool.master.version"
> and "my.cool.alternate.master.version" and set both to the same value
> of "1.2-SNAPSHOT".
>
> In szenario 1: I hard code the version of the master to "1.2-SNAPSHOT"
> but use the property to reference the parent from the moule ... when
> running a build, maven tries to download
> "de/mycompany/test/${my.cool.master.version}/mycoolmaster-${my.cool.master.version}.pom"
> --> Failure, because the property is not resolved
>
> In szenario 2: I use the same variable for defining the masters
> version. This time the maven build runs fine and the parent version is
> correctly resolved.
>
> In szenario 3: I use the first property to define the version of the
> master and the second one for referncing the parent from the module
> ... when running a build, maven tries to download
> "de/mycompany/test/${my.cool.alternate.master.version}/mycoolmaster-${my.cool.alternate.master.version}.pom"
> --> Failure, because the property is not resolved
>
> So to me it looks as if there was some sort of intention behind
> everything and not a bug in the system as I was told on the dev-list.
> To me it looks like one teeniewienie loophole allowing properies in
> versions while closing the usage range so much that possible harm is
> reduced to it's absolute minimum. So it seems that my usecase seems to
> be the onlly one allowed. After all ... this is a problem users are
> begging for maven to provide a solution since maven 2.0 (When looking
> at the forums).
>
> Ok ... and now to finish the loop back to my topic:
>
> If I am doing bad sourcery ... how would I setup one maven build to
> allow simple releases of individual modules with individual versions?
>
> Chris
>
> [ C h r i s t o f e r D u t z ]
>
> C-Ware IT-Service
>
> Inhaber
>
> Dipl. Inf. Christofer Dutz
>
> Karlstraße. 104, 64285 Darmstadt
>
> Beschreibung: 788335
> <http://www.benchpark.com/788335/kundenzufriedenheit.htm>
>
> IT- und Systemhäuser <http://www.benchpark.com/it_und_systemhaeuser.htm>
>
> fon: 0 61 51 / 27315 - 61
>
> fax: 0 61 51 / 27315 - 64
>
> mobil: 0171 / 7 444 2 33
>
> email: christofer.dutz@c-ware.de <ma...@c-ware.de>
>
> http://www.c-ware.de <http://www.c-ware.de/>
>
> UStId-Nr. DE195700962
>
--
Ron Wheeler
President
Artifact Software Inc
email: rwheeler@artifact-software.com
skype: ronaldmwheeler
phone: 866-970-2435, ext 102