You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@maven.apache.org by CD...@behr.com on 2008/05/29 17:42:09 UTC
How to better manage cascading releases
We have migrated all of our internal code to Maven now. We have many
separate top-level projects and many of these projects are multi-module
projects. We have done a good job of moving common code elements out to
their own project so they can easily be reused. Generally speaking from a
basic object-oriented architecture standpoint, we are happy with how we
have architected our code.
However after all of that good work, we have run in to a slight drawback
when it comes time to make releases of our code. There is no way we are
the biggest code base out there using Maven, so I am hoping someone can
help us with this issue.
Imagine we have Project-A - it is a multi-module project and the main
artifact it produces is a JavaEE application packaged as an Ear file.
However this project depends on our shared company-wide uberpom, it also
depends on 3 or 4 Java services that are in their own projects and it also
depends on our internal commons library also in its own project. Depending
on what has happened since the last release, Project-A may be depending on
SNAPSHOT versions of some or all of these other Projects. So to make a
release we have to first make releases of the other Projects (and also any
SNAPSHOTS they depend on etc. etc.). Then as we work our way back up the
dependency tree we have to edit pom files to change the SNAPSHOT
dependencies to the just released version numbers. And finally we can
release Project-A. This is a very time consuming process for us. This is a
very real world problem, we recently did a release that ended up causing
us to release 13 separate projects.
So there seems to be 3 possible conclusions:
- There is any easy way to make these kinds of cascading releases, but we
just don't know about it
- There is a better way to structure our Maven projects so this doesn't
happen in the first place (keeping in mind that we are happy with how the
code itself is architected currently)
- This is just the way it is, so we should get used to it
Can anyone comment on this?
Thanks
------------------------------------------------------------------------------
Craig Dickson
Software Engineering Manager
Behr Process Corporation
Santa Ana, California
_________________________
The information contained in this e-mail message may be proprietary,
privileged, confidential or protected from disclosure. If you are not the
intended recipient, any dissemination, distribution or copying is strictly
prohibited. If you think that you have received this e-mail message in
error, please e-mail the sender.
Re: How to better manage cascading releases
Posted by Geoffrey Wiseman <ge...@gmail.com>.
On Thu, May 29, 2008 at 11:42 AM, <CD...@behr.com> wrote:
> So there seems to be 3 possible conclusions:
At least, yeah.
> - There is any easy way to make these kinds of cascading releases, but we
> just don't know about it
I don't know of one, but if there is, happy to hear it.
> - There is a better way to structure our Maven projects so this doesn't
> happen in the first place (keeping in mind that we are happy with how the
> code itself is architected currently)
Probably partially true. Some things I've considered:
- When it's entirely internal, possibly living with a snapshot version.
I don't like this, because it reduces the reproducibility, but it would
simplify.
- Try and ensure that most common code is extensible to the point that I
don't need to modify the upstream projects frequently. So, for instance, if
I had a common web-testing framework for the company that was in 1.1
(web-testing-1.1) used by my enterprise project (enterprise-project-1.0),
and I wanted to alter a class in web-testing, I could do so with a subclass
that's specific to enterprise-project-1.0. I could release e-p-1.0 without
releasing a new version of w-t, and when I feel like the change is stable
and ready to migrate upward, I could release a w-t-1.2 release at my
leisure, then migrate e-p to take advantage.
I'm sure there are other possibilities in this vein.
- This is just the way it is, so we should get used to it
If you can live with that answer, my first instinct is that it's at least
partially true, yeah. Curious to see other responses.
- Geoffrey
--
Geoffrey Wiseman
Re: How to better manage cascading releases
Posted by Benoit Decherf <de...@yahoo-inc.com>.
I integrate a similar process in continuum :
http://jira.codehaus.org/browse/CONTINUUM-1633
I'm still working on it.
Benoit
Ken Liu wrote:
> This is a great question...I have wondered the same thing myself.
>
> I am working on a complex project that involves simultaneously releasing 6
> ears. The whole build consists of 25+ maven projects (not counting
> modules) that are built and released together (relax, many of them are
> purely assemblies). The dependency tree between all of these projects is at
> least 5 deep.
>
> The way we have solved the problem is to always declare SNAPSHOT
> dependencies (for internal dependencies). At release time, there is a
> script that tags the trunk, checks out the tag, updates the pom versions,
> updates the scm path, updates the dependency versions (changes the SNAPSHOTs
> to released numbers), then commits the changes and builds the release. This
> is basically the same as what the release-plugin does. The script does this
> once for each project in the correct dependency order.
>
> The script is driven by a yml file that is the metadata for the "super
> build" - it contains the list of projects, svn locations, and target version
> numbers, and dependencies between projects. The script is around 200 lines
> of ruby code.
>
> The script is run manually, which is ok since we do this "official release"
> only once every few weeks. (Each individual project is built in Continuum
> during normal development.)
>
> I justified the cost/time of developing these scripts to my manager by
> explaining that it took me around two days in the beginning to do the entire
> build, now it takes about two hours (some artifacts take a long time to run
> the unit tests) from start to finish. I spent a little bit of time before
> each release working on the script so now there is a net cost savings :)
>
> AFAIK, there is no easy way to do this. I suspect many organizations don't
> do anything so complex and others do not attempt to _simultaneously_ release
> so many projects, so the cost of releasing is spread out, and is not
> noticed.
>
> Ken
>
>
> On 5/31/08, Stephen Connolly <st...@gmail.com> wrote:
>
>> We have a four uber project set of releases.
>>
>> We use scripts to check out the latest released pom.xml for each from svn,
>> pull out the version number, step back if it's a -SNAPSHOT, update our
>> module's dependencies to this version, run maven with the integration tests
>> and check in the updated pom.xml if the tests pass before doing mvn
>> release:prepare release:perform -B
>>
>> This is set up as a number of release jobs in Hudson.
>>
>> Just kick off a build and you get the release when it's all done, or an
>> email telling you what broke...
>>
>> I would like to do a bit more work and make it keep trying less new
>> versions
>> of the dependency until it falls back to the one it started with... but
>> that's getting too fancy... the script is currently only 100 or so lines
>> long
>>
>> -Stephen
>>
>> On Fri, May 30, 2008 at 9:12 AM, Bracewell, Robert <rb...@qualcomm.com>
>> wrote:
>>
>>
>>> In our organization it depends on the project but I have projects that
>>> release twice a week internally. Other groups or projects that are
>>> reliant on such artifacts can then decide as and when they want to
>>> depend on the new artifacts that were deployed.
>>>
>>> -----Original Message-----
>>> From: Geoffrey Wiseman [mailto:geoffrey.wiseman@gmail.com]
>>> Sent: 30 May 2008 03:35
>>> To: Maven Users List
>>> Subject: Re: How to better manage cascading releases
>>>
>>> On Thu, May 29, 2008 at 6:39 PM, Michael McCallum <gh...@apache.org>
>>> wrote:
>>>
>>>
>>>> release early release often... we don't use snapshot dependencies and
>>>> release
>>>> artifacts early. So if you are working on one of the 13 dependent
>>>>
>>> libraries
>>>
>>>> as soon as you - the dev - is happy the change is ready for use then
>>>>
>>> you
>>>
>>>> release it. why leave it as a snapshot? If the change would break
>>>>
>>> anything
>>>
>>>> useing it we bump the major version up so its not pulled in until
>>>> downstream
>>>> users are ready.
>>>>
>>>> if you use version ranges and manage codelines by major version then
>>>>
>>> you
>>>
>>>> can
>>>> easily have the trunk of a project being actively developed and
>>>>
>>> released
>>>
>>>> without pulling it into a deliverable.
>>>>
>>> Hmm, interesting perspective.
>>>
>>> I still find it takes an hour or two to pull off a release, between the
>>> dry-run, the actual prepare and the perform -- do you find that cost
>>> goes
>>> down if you release a lot, or have tricks for reducing the cost of
>>> releasing?
>>>
>>> - Geoffrey
>>> --
>>> Geoffrey Wiseman
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
>>> For additional commands, e-mail: users-help@maven.apache.org
>>>
>>>
>>>
>
>
Re: How to better manage cascading releases
Posted by Ken Liu <ke...@gmail.com>.
This is a great question...I have wondered the same thing myself.
I am working on a complex project that involves simultaneously releasing 6
ears. The whole build consists of 25+ maven projects (not counting
modules) that are built and released together (relax, many of them are
purely assemblies). The dependency tree between all of these projects is at
least 5 deep.
The way we have solved the problem is to always declare SNAPSHOT
dependencies (for internal dependencies). At release time, there is a
script that tags the trunk, checks out the tag, updates the pom versions,
updates the scm path, updates the dependency versions (changes the SNAPSHOTs
to released numbers), then commits the changes and builds the release. This
is basically the same as what the release-plugin does. The script does this
once for each project in the correct dependency order.
The script is driven by a yml file that is the metadata for the "super
build" - it contains the list of projects, svn locations, and target version
numbers, and dependencies between projects. The script is around 200 lines
of ruby code.
The script is run manually, which is ok since we do this "official release"
only once every few weeks. (Each individual project is built in Continuum
during normal development.)
I justified the cost/time of developing these scripts to my manager by
explaining that it took me around two days in the beginning to do the entire
build, now it takes about two hours (some artifacts take a long time to run
the unit tests) from start to finish. I spent a little bit of time before
each release working on the script so now there is a net cost savings :)
AFAIK, there is no easy way to do this. I suspect many organizations don't
do anything so complex and others do not attempt to _simultaneously_ release
so many projects, so the cost of releasing is spread out, and is not
noticed.
Ken
On 5/31/08, Stephen Connolly <st...@gmail.com> wrote:
>
> We have a four uber project set of releases.
>
> We use scripts to check out the latest released pom.xml for each from svn,
> pull out the version number, step back if it's a -SNAPSHOT, update our
> module's dependencies to this version, run maven with the integration tests
> and check in the updated pom.xml if the tests pass before doing mvn
> release:prepare release:perform -B
>
> This is set up as a number of release jobs in Hudson.
>
> Just kick off a build and you get the release when it's all done, or an
> email telling you what broke...
>
> I would like to do a bit more work and make it keep trying less new
> versions
> of the dependency until it falls back to the one it started with... but
> that's getting too fancy... the script is currently only 100 or so lines
> long
>
> -Stephen
>
> On Fri, May 30, 2008 at 9:12 AM, Bracewell, Robert <rb...@qualcomm.com>
> wrote:
>
> > In our organization it depends on the project but I have projects that
> > release twice a week internally. Other groups or projects that are
> > reliant on such artifacts can then decide as and when they want to
> > depend on the new artifacts that were deployed.
> >
> > -----Original Message-----
> > From: Geoffrey Wiseman [mailto:geoffrey.wiseman@gmail.com]
> > Sent: 30 May 2008 03:35
> > To: Maven Users List
> > Subject: Re: How to better manage cascading releases
> >
> > On Thu, May 29, 2008 at 6:39 PM, Michael McCallum <gh...@apache.org>
> > wrote:
> >
> > > release early release often... we don't use snapshot dependencies and
> > > release
> > > artifacts early. So if you are working on one of the 13 dependent
> > libraries
> > > as soon as you - the dev - is happy the change is ready for use then
> > you
> > > release it. why leave it as a snapshot? If the change would break
> > anything
> > > useing it we bump the major version up so its not pulled in until
> > > downstream
> > > users are ready.
> > >
> > > if you use version ranges and manage codelines by major version then
> > you
> > > can
> > > easily have the trunk of a project being actively developed and
> > released
> > > without pulling it into a deliverable.
> >
> >
> > Hmm, interesting perspective.
> >
> > I still find it takes an hour or two to pull off a release, between the
> > dry-run, the actual prepare and the perform -- do you find that cost
> > goes
> > down if you release a lot, or have tricks for reducing the cost of
> > releasing?
> >
> > - Geoffrey
> > --
> > Geoffrey Wiseman
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> > For additional commands, e-mail: users-help@maven.apache.org
> >
> >
>
Re: How to better manage cascading releases
Posted by Stephen Connolly <st...@gmail.com>.
We have a four uber project set of releases.
We use scripts to check out the latest released pom.xml for each from svn,
pull out the version number, step back if it's a -SNAPSHOT, update our
module's dependencies to this version, run maven with the integration tests
and check in the updated pom.xml if the tests pass before doing mvn
release:prepare release:perform -B
This is set up as a number of release jobs in Hudson.
Just kick off a build and you get the release when it's all done, or an
email telling you what broke...
I would like to do a bit more work and make it keep trying less new versions
of the dependency until it falls back to the one it started with... but
that's getting too fancy... the script is currently only 100 or so lines
long
-Stephen
On Fri, May 30, 2008 at 9:12 AM, Bracewell, Robert <rb...@qualcomm.com>
wrote:
> In our organization it depends on the project but I have projects that
> release twice a week internally. Other groups or projects that are
> reliant on such artifacts can then decide as and when they want to
> depend on the new artifacts that were deployed.
>
> -----Original Message-----
> From: Geoffrey Wiseman [mailto:geoffrey.wiseman@gmail.com]
> Sent: 30 May 2008 03:35
> To: Maven Users List
> Subject: Re: How to better manage cascading releases
>
> On Thu, May 29, 2008 at 6:39 PM, Michael McCallum <gh...@apache.org>
> wrote:
>
> > release early release often... we don't use snapshot dependencies and
> > release
> > artifacts early. So if you are working on one of the 13 dependent
> libraries
> > as soon as you - the dev - is happy the change is ready for use then
> you
> > release it. why leave it as a snapshot? If the change would break
> anything
> > useing it we bump the major version up so its not pulled in until
> > downstream
> > users are ready.
> >
> > if you use version ranges and manage codelines by major version then
> you
> > can
> > easily have the trunk of a project being actively developed and
> released
> > without pulling it into a deliverable.
>
>
> Hmm, interesting perspective.
>
> I still find it takes an hour or two to pull off a release, between the
> dry-run, the actual prepare and the perform -- do you find that cost
> goes
> down if you release a lot, or have tricks for reducing the cost of
> releasing?
>
> - Geoffrey
> --
> Geoffrey Wiseman
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
>
RE: How to better manage cascading releases
Posted by "Bracewell, Robert" <rb...@qualcomm.com>.
In our organization it depends on the project but I have projects that
release twice a week internally. Other groups or projects that are
reliant on such artifacts can then decide as and when they want to
depend on the new artifacts that were deployed.
-----Original Message-----
From: Geoffrey Wiseman [mailto:geoffrey.wiseman@gmail.com]
Sent: 30 May 2008 03:35
To: Maven Users List
Subject: Re: How to better manage cascading releases
On Thu, May 29, 2008 at 6:39 PM, Michael McCallum <gh...@apache.org>
wrote:
> release early release often... we don't use snapshot dependencies and
> release
> artifacts early. So if you are working on one of the 13 dependent
libraries
> as soon as you - the dev - is happy the change is ready for use then
you
> release it. why leave it as a snapshot? If the change would break
anything
> useing it we bump the major version up so its not pulled in until
> downstream
> users are ready.
>
> if you use version ranges and manage codelines by major version then
you
> can
> easily have the trunk of a project being actively developed and
released
> without pulling it into a deliverable.
Hmm, interesting perspective.
I still find it takes an hour or two to pull off a release, between the
dry-run, the actual prepare and the perform -- do you find that cost
goes
down if you release a lot, or have tricks for reducing the cost of
releasing?
- Geoffrey
--
Geoffrey Wiseman
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: How to better manage cascading releases
Posted by Geoffrey Wiseman <ge...@gmail.com>.
On Thu, May 29, 2008 at 6:39 PM, Michael McCallum <gh...@apache.org> wrote:
> release early release often... we don't use snapshot dependencies and
> release
> artifacts early. So if you are working on one of the 13 dependent libraries
> as soon as you - the dev - is happy the change is ready for use then you
> release it. why leave it as a snapshot? If the change would break anything
> useing it we bump the major version up so its not pulled in until
> downstream
> users are ready.
>
> if you use version ranges and manage codelines by major version then you
> can
> easily have the trunk of a project being actively developed and released
> without pulling it into a deliverable.
Hmm, interesting perspective.
I still find it takes an hour or two to pull off a release, between the
dry-run, the actual prepare and the perform -- do you find that cost goes
down if you release a lot, or have tricks for reducing the cost of
releasing?
- Geoffrey
--
Geoffrey Wiseman
Re: How to better manage cascading releases
Posted by Michael McCallum <gh...@apache.org>.
On Fri, 30 May 2008 03:42:09 CDickson@behr.com wrote:
>
> So there seems to be 3 possible conclusions:
> - There is any easy way to make these kinds of cascading releases, but we
> just don't know about it
> - There is a better way to structure our Maven projects so this doesn't
> happen in the first place (keeping in mind that we are happy with how the
> code itself is architected currently)
> - This is just the way it is, so we should get used to it
>
> Can anyone comment on this?
release early release often... we don't use snapshot dependencies and release
artifacts early. So if you are working on one of the 13 dependent libraries
as soon as you - the dev - is happy the change is ready for use then you
release it. why leave it as a snapshot? If the change would break anything
useing it we bump the major version up so its not pulled in until downstream
users are ready.
if you use version ranges and manage codelines by major version then you can
easily have the trunk of a project being actively developed and released
without pulling it into a deliverable.
--
Michael McCallum
Enterprise Engineer
mailto:gholam@apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org