Best practice for migrating existing system

[David McCabe <da...@sunlightd.com>]

We've been using a custom-made component-based build system for many years,
built using a number of Ant scripts, and we'd really like to move to
something more standard. We'd like to go with Maven, Subversion and Nexus,
but our build processes are a little different from the usual and it would
be nice to know what is the 'right' way to address some of the versioning
issues.

Our applications are made up of a few dozen components, some of which are
shared between projects and some of which are project-specific. Each of
these components currently has a 4-component build number
"major.minor.patch.build". When building a component, its build number is
incremented. A new major, minor or patch version resets the build version to
zero. I'm happy to change a dot to a hyphen, and I'd also be perfectly happy
for that build number to be a Subversion revision number, but I'm not happy
to go for a SNAPSHOT-style date/time: try telling someone your latest
version is "3.6.0.20110507170144".

The 'released' version of a component is the last build number that
component is built with. Once a component has been 'released' it cannot be
built as a new version; the rest of the version number needs to be
incremented. This is somewhat like Maven's release process except that we do
not want to renumber or rebuild the components for release; integration and
system tests will reveal when a project (and its constituent components) are
ready for release, and we're not going to re-run the entire test suite on a
functionally identical build. It would be nice to protect a 'released'
component from being rebuilt but that's hardly mandatory.

Obviously, when a component is built and its build number incremented, all
dependencies need to have their version numbers updated and be rebuilt. It
would be nice if the dependencies could automatically pick up the 'latest'
version of a major.minor.patch dependency version; I understand this can be
done using Maven version ranges.

When doing a 'real' build I would like the components to be 'tagged' in
Subversion with the built version numbers and for that tag to reference the
actual set of build dependencies (rather than a version range). Ideally, I'd
have a Subversion style:

Component-name
|- trunk
|- branches
|- |- 1.0.0
|- |- 1.0.1
|- |- 1.1.0
|- tags
|- |- 1.0.0-0
|- |- 1.0.0-1
|- |- 1.0.1-0

I would also like to be able to do 'test' builds without tagging; maybe the
tagging should be separate so I can try to build the whole lot to my local
repository, and if the build is successful and all automated tests pass
along the way only then update Nexus and tag in Subversion.

What I want, therefore, is a system in which, for a set of top-level
components to be built, it would then evaluate the set of transitive
dependencies of those components as specified in Subversion (not Nexus or
the local repository), work out the complete list of out-of-date components
to build and then build all of those components such that the top-level
components represent the current state of Subversion. Plainly, if the build
is up-to-date with Subversion, nothing should be built.

I accept that this is a job for Python, but I am wondering how much of this
process can be done with Maven without cluttering the per-project POMs, even
if minor modifications need to be made.

Thanks in advance,
David McCabe


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org

Re: Best practice for migrating existing system

[Ron Wheeler <rw...@artifact-software.com>]

On 07/05/2011 6:59 PM, David McCabe wrote:
> Thanks for the information. I think I may need to clarify some things.
>
>>> Our applications are made up of a few dozen components, some of which
>>> are shared between projects and some of which are project-specific.
>>> Each of these components currently has a 4-component build number
>>> "major.minor.patch.build". When building a component, its build number
>>> is incremented. A new major, minor or patch version resets the build
>>> version to zero. I'm happy to change a dot to a hyphen, and I'd also
>>> be perfectly happy for that build number to be a Subversion revision
>>> number, but I'm not happy to go for a SNAPSHOT-style date/time: try
>>> telling someone your latest version is "3.6.0.20110507170144".
>> A SNAPSHOT is not a release and ideally should never be spoken off outside
>> the immediate team and even then you need to explain what a SNAPSHOT
>> does and does not yet do and what are the restrictions on its use and
> where
>> it does not respect the specifications. Or you can just let the other team
>> members curse you out when they spend hours trying to find out why their
>> software does not work when they use your SNAPSHOTs.
>>
>> SNAPSHOTS are very useful but version 1.3.5.6 is not the same as 1.3.5.6-
>> SNAPSHOT.
>> One comes with a warranty and the other comes with a warning.
> Absolutely, but therein lies the problem. If we were to use SNAPSHOT, it
> would be 1.3.5-SNAPSHOT. In Maven terminology, this would result in you
> eventually 'releasing' a 1.3.5-SNAPSHOT as version 1.3.5.
>
> Our problem is that the builds that get tested are, presumably, SNAPSHOT
> builds of each of the components modified since the last project release.
> The 'Maven way' of doing this appears to be for every candidate for project
> release, I 'release' every component.
You deploy a SNAPSHOT until you are ready to release 1.3.5 or release 
1.3.5-RC1 or you want to create a release candidate. We don't use this 
but I think that a lot of people do this.
> If the testing fails (or a product
> manager changes his mind about how a feature should work, e.g. from beta
> feedback) then we would wind up 'releasing' each component modified in
> making that change before we had a build that could again be a 'release
> candidate'. Is that correct? This seems to make Maven 'release' more of a
> 'make-release-candidate' - and really quite time-consuming.
>
I think that you are right here.
We have made sure that our modules are functional and are working 
towards better SOA with every release so we do not have a lot of 
functions that cross modules.
Changes tend to be local once we get close to releases.
Using Betas and RCs will help if you are prone to late cycle changes in 
functionality.

It is not that hard to change the version ID in each POM. Takes a few 
seconds for each POM. You can automate this with the maven release  
plug-in if you think that this is too much work to do this manually. We 
don't do this but you can get others in the forum to guide you in this.

> This isn't a Web application - this is a 60-component medical imaging
> workstation, and the component dependency graph is deep rather than broad. A
> change can be only a line of code but still result in a rebuild of many
> components.
>
Doesn't change things very much.
Not sure why a single line of code should result in rebuilding more than 
1 module.
You will have to rebuild the assembly of the application so I see only 2 
modules being rebuilt in your example.
> Maybe the idea is that one can 'release' all of the components after
> testing? My major concern is trying to convince our quality team that having
> built the application against one set of versions that another set of
> versions are fundamentally identical.
You test the release after you build it. One hopes that the release is 
the same as the last release candidate and only requires some very 
limited functional user testing after the automated tests are run.
> Is there some way for me to check that
> a component's build-time transitive dependencies' versions were all
> 'released'?
>
Only use releases versions of dependencies in your build. You will get 
what you specify. If the POM that builds your run-time application calls 
for 1.3.5 of dependency A, then you are sure that your application is 
using version 1.3.5 of jar A.
>>> What I want, therefore, is a system in which, for a set of top-level
>>> components to be built, it would then evaluate the set of transitive
>>> dependencies of those components as specified in Subversion (not Nexus
I don't think so. You tag and branch the Subversion versions based on 
the things that you build.
>> or
>>> the local repository), work out the complete list of out-of-date
>> components
>>> to build and then build all of those components such that the top-level
>>> components represent the current state of Subversion. Plainly, if the
> build
>>> is up-to-date with Subversion, nothing should be built.
>>>
>> Too much work. Just run a clean development workflow and do things the
>> Maven way and your life will be much more pleasant and your development
>> team much happier with each other.
>> Guys who release crap will soon find out that their friends will jump on
>> them and the boss will know that a crappy piece of software was released
>> when it was still at the SNAPSHOT stage.
> Alas, this is a large project but not a large team, and many components do
> not have single ownership because there just aren't enough people to do
> that. There are five of us across three projects and the component
> versioning process is a means to allow us to manage this size of project. I
> have to be absolutely sure that each check-in that any of us made has made
> it to the installer. I currently do this manually, and it's very
> time-consuming and error-prone (particularly since we're on CVS, so I can't
> just look at the revision log since the last build...). If a change has
> occurred a long way down the dependency tree, it needs a way to propagate
> upwards.
>
We are a team of 2 but used to be 3 with over 80 modules making up 2 
distinct applications with no code overlap.
Ownership is not an issue if you use Subversion, Nexus and Maven 
properly and have a good modular design.
The POM file with its description of the modules dependencies is the 
gospel over Subversion.

If the POM says that you built version 1.3.5.1 of your module A with 
version 1.3.5.0 of dependency B, that is what you did.
You don't care what Subversion says about your modules A or B.

In your current system, you spend a lot of effort controlling code 
versions because your build system does not care about versions so you 
are forced to look at Subversion or CVS to control versions.
Once your build system (Maven) starts to care about versions and 
guarantees that builds use the right versions of all your dependencies, 
your life gets a lot easier.
You can commit anything to Subversion that you want to share, regardless 
of the state of the code because anyone who checks out a module can see 
from the POM which version the code is supposed to be building.
If you build a release or SNAPSHOT using  your Nexus dependencies (My 
advice is to never build with a workspace artifact.)  you will know 
exactly what you are getting regardless of what state the Subversion 
code is in.

Even if you forget to tag the Subversion version, you can find the right 
version of the code by looking at the Subversion history to see when the 
POM changed.
It is easier if you handle Subversion right but not fatal if you don't.
Getting people to put useful comments on every commit to Subversion is 
really helpful.

SNAPSHOTS are a powerful tool to help a team work together.
Stubs, partial implementations can be shared as SNAPSHOTS.
The person providing the SNAPSHOT and the person using it need to 
communicate so that as the SNAPSHOT dependency gets more functionality 
and fewer bugs, the person writing the code that depends on it is aware 
of the state.

> I am trying to find out what the 'Maven way' is for each of these issues,
> but there has to be flexibility. Maven is the closest thing out there to the
> system we have right now, and I would much prefer to use it than to bolt on
> additional functionality to our large Ant script collection... but if it
> won't fit our organization, then it is Maven that will go.
>
Ant and Maven are based on completely different philosophies.

I don't see anything yet that you are doing or building that could not 
be handled by Maven.
Thousands of teams are building applications like yours.

Try to adopt the simplest Maven approach and then add complexity later 
if you see a clear way to optimize your workflow.
Trying to optimize your use of Maven at the start can get you into a 
complex and poorly structured set of Maven modules with too many obscure 
plug-ins.
My impression is that you need to use the compile plug-in and the 
assembly plug-in to build the Desktop jar.

My advice is to start out by taking the time to manually manage the 
updating of the POM versions until you get comfortable with the workflow 
and then see how this can be automated or simplified.
We opted for simplification over automation and put in the time up front 
in a new release to plan how the versions of all of the modules that 
make up the application will be affected by the proposed changes.
We have decided to treat our own modules in the same way we treat third 
party modules.
We do not force all of the modules to have the same version number.
If module B is not affected by the change, version 1.3.5.1  of module A 
can depend on version 1.3.5.0 of module B. We start with module A 
version 1.3.5.1-SNAPSHOT depending in release 1.3.5.0 of module B.
This reduced the effort required to manage the project when adding new 
features or fixing bugs which have limited scope.

I hope that this helps you give you the confidence to move forward.

Ron
> Thanks,
> David
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org

RE: Best practice for migrating existing system

[David McCabe <da...@sunlightd.com>]

Thanks for the information. I think I may need to clarify some things.

> > Our applications are made up of a few dozen components, some of which
> > are shared between projects and some of which are project-specific.
> > Each of these components currently has a 4-component build number
> > "major.minor.patch.build". When building a component, its build number
> > is incremented. A new major, minor or patch version resets the build
> > version to zero. I'm happy to change a dot to a hyphen, and I'd also
> > be perfectly happy for that build number to be a Subversion revision
> > number, but I'm not happy to go for a SNAPSHOT-style date/time: try
> > telling someone your latest version is "3.6.0.20110507170144".
> A SNAPSHOT is not a release and ideally should never be spoken off outside
> the immediate team and even then you need to explain what a SNAPSHOT
> does and does not yet do and what are the restrictions on its use and
where
> it does not respect the specifications. Or you can just let the other team
> members curse you out when they spend hours trying to find out why their
> software does not work when they use your SNAPSHOTs.
> 
> SNAPSHOTS are very useful but version 1.3.5.6 is not the same as 1.3.5.6-
> SNAPSHOT.
> One comes with a warranty and the other comes with a warning.

Absolutely, but therein lies the problem. If we were to use SNAPSHOT, it
would be 1.3.5-SNAPSHOT. In Maven terminology, this would result in you
eventually 'releasing' a 1.3.5-SNAPSHOT as version 1.3.5.

Our problem is that the builds that get tested are, presumably, SNAPSHOT
builds of each of the components modified since the last project release.
The 'Maven way' of doing this appears to be for every candidate for project
release, I 'release' every component. If the testing fails (or a product
manager changes his mind about how a feature should work, e.g. from beta
feedback) then we would wind up 'releasing' each component modified in
making that change before we had a build that could again be a 'release
candidate'. Is that correct? This seems to make Maven 'release' more of a
'make-release-candidate' - and really quite time-consuming.

This isn't a Web application - this is a 60-component medical imaging
workstation, and the component dependency graph is deep rather than broad. A
change can be only a line of code but still result in a rebuild of many
components.

Maybe the idea is that one can 'release' all of the components after
testing? My major concern is trying to convince our quality team that having
built the application against one set of versions that another set of
versions are fundamentally identical. Is there some way for me to check that
a component's build-time transitive dependencies' versions were all
'released'?

> > What I want, therefore, is a system in which, for a set of top-level
> > components to be built, it would then evaluate the set of transitive
> > dependencies of those components as specified in Subversion (not Nexus
> or
> > the local repository), work out the complete list of out-of-date
> components
> > to build and then build all of those components such that the top-level
> > components represent the current state of Subversion. Plainly, if the
build
> > is up-to-date with Subversion, nothing should be built.
> >
> Too much work. Just run a clean development workflow and do things the
> Maven way and your life will be much more pleasant and your development
> team much happier with each other.
> Guys who release crap will soon find out that their friends will jump on
> them and the boss will know that a crappy piece of software was released
> when it was still at the SNAPSHOT stage.

Alas, this is a large project but not a large team, and many components do
not have single ownership because there just aren't enough people to do
that. There are five of us across three projects and the component
versioning process is a means to allow us to manage this size of project. I
have to be absolutely sure that each check-in that any of us made has made
it to the installer. I currently do this manually, and it's very
time-consuming and error-prone (particularly since we're on CVS, so I can't
just look at the revision log since the last build...). If a change has
occurred a long way down the dependency tree, it needs a way to propagate
upwards.

I am trying to find out what the 'Maven way' is for each of these issues,
but there has to be flexibility. Maven is the closest thing out there to the
system we have right now, and I would much prefer to use it than to bolt on
additional functionality to our large Ant script collection... but if it
won't fit our organization, then it is Maven that will go.

Thanks,
David


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org

Re: Best practice for migrating existing system

[Wayne Fay <wa...@gmail.com>]

> of 'common' component that (a) gets a lot of changes and (b) has many
> things that depend on it -- at least in the early phases of building
> out a large system. Over time, that component slows down, or even
> splits up.

Yes, this echoes my experience as well.

Wayne

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org

Re: Best practice for migrating existing system

[Benson Margulies <bi...@gmail.com>]

Not every problem is solved by waving the lemon of 'SOA' over it and
expecting it to turn into lemonade. Some of us code things that have
actual computations in them, and those computations have shared,
reused classes, and those classes do not belong behind a web service
-- either for performance or sheer common sense.

And the effect of this, in toto, is that it highlights some of the
edges of what the maven-release-plugin is useful for. When working on
the far side of those edges, it makes (to me) more sense to find
another procedure than to warp the software architecture.

There's a line between "the maven way" and "procrustes' bed," and that
line does not necessarily pass through a point at infinity.

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org

Re: Best practice for migrating existing system

[Ron Wheeler <rw...@artifact-software.com>]

On 09/05/2011 11:25 AM, Benson Margulies wrote:
> It sounds good, but there's a medium size in which it doesn't quite
> work out this way. You can't create a new maven module for every
> single class. Thus, in my experience, we have ended up with some sort
> of 'common' component that (a) gets a lot of changes and (b) has many
> things that depend on it -- at least in the early phases of building
> out a large system. Over time, that component slows down, or even
> splits up.
Every class is way too much.

Separating business functions from utilities is a good place to start.

Moving to SOA and breaking the business processes into services is a 
"good thing".
This hides datastructure details from many levels and isolates 
processing into services that are shared.
Functional changes start to have smaller scope and it becomes easier to 
identify the modules that are affected.

Ron
> On Mon, May 9, 2011 at 11:10 AM, Ron Wheeler
> <rw...@artifact-software.com>  wrote:
>> On 09/05/2011 10:42 AM, Wayne Fay wrote:
>>>> As far as I can tell, Option 1 is only practical once the velocity of
>>>> change slows down to the point where at least some components sit
>>>> unchanged for a good long time -- that is, for more than one
>>>> development iteration. It's only useful to 'use a version with a
>>> But it seems like a good (best?) practice to break modules up into
>>> functional components such that parts which have no reason to change
>>> for a given release are not changed. If you find it annoying that a
>>> small change in a small piece of code that is wrapped up inside a much
>>> larger library means the entire library has to bump a version, you may
>>> want to consider breaking the smaller bits out into its own
>>> independent module. Especially if the velocity of change in those
>>> small bits is significantly greater than the velocity of the rest of
>>> the bits.
>>>
>>> This does mean that you may end up with 150 modules rather than 70.
>>> Suddenly your processes which were OK at 70 modules are really not
>>> working with 150, so you may need to be willing to adjust your
>>> processes etc as well.
>> +1
>> More robust structure
>> Easier to assure that small functional modules are working correctly
>> Easier to test libraries than applications - usually.
>> Increases likeliness of sharing of code.
>>
>> Lots of other good reasons to do this.
>>
>> Maven and Nexus makes it easy to manage this.
>>
>> Ron
>>> Wayne
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
>>> For additional commands, e-mail: users-help@maven.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
>> For additional commands, e-mail: users-help@maven.apache.org
>>
>>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org

Re: Best practice for migrating existing system

[Benson Margulies <bi...@gmail.com>]

It sounds good, but there's a medium size in which it doesn't quite
work out this way. You can't create a new maven module for every
single class. Thus, in my experience, we have ended up with some sort
of 'common' component that (a) gets a lot of changes and (b) has many
things that depend on it -- at least in the early phases of building
out a large system. Over time, that component slows down, or even
splits up.

On Mon, May 9, 2011 at 11:10 AM, Ron Wheeler
<rw...@artifact-software.com> wrote:
> On 09/05/2011 10:42 AM, Wayne Fay wrote:
>>>
>>> As far as I can tell, Option 1 is only practical once the velocity of
>>> change slows down to the point where at least some components sit
>>> unchanged for a good long time -- that is, for more than one
>>> development iteration. It's only useful to 'use a version with a
>>
>> But it seems like a good (best?) practice to break modules up into
>> functional components such that parts which have no reason to change
>> for a given release are not changed. If you find it annoying that a
>> small change in a small piece of code that is wrapped up inside a much
>> larger library means the entire library has to bump a version, you may
>> want to consider breaking the smaller bits out into its own
>> independent module. Especially if the velocity of change in those
>> small bits is significantly greater than the velocity of the rest of
>> the bits.
>>
>> This does mean that you may end up with 150 modules rather than 70.
>> Suddenly your processes which were OK at 70 modules are really not
>> working with 150, so you may need to be willing to adjust your
>> processes etc as well.
>
> +1
> More robust structure
> Easier to assure that small functional modules are working correctly
> Easier to test libraries than applications - usually.
> Increases likeliness of sharing of code.
>
> Lots of other good reasons to do this.
>
> Maven and Nexus makes it easy to manage this.
>
> Ron
>>
>> Wayne
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
>> For additional commands, e-mail: users-help@maven.apache.org
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org

Re: Best practice for migrating existing system

[Ron Wheeler <rw...@artifact-software.com>]

On 09/05/2011 10:42 AM, Wayne Fay wrote:
>> As far as I can tell, Option 1 is only practical once the velocity of
>> change slows down to the point where at least some components sit
>> unchanged for a good long time -- that is, for more than one
>> development iteration. It's only useful to 'use a version with a
> But it seems like a good (best?) practice to break modules up into
> functional components such that parts which have no reason to change
> for a given release are not changed. If you find it annoying that a
> small change in a small piece of code that is wrapped up inside a much
> larger library means the entire library has to bump a version, you may
> want to consider breaking the smaller bits out into its own
> independent module. Especially if the velocity of change in those
> small bits is significantly greater than the velocity of the rest of
> the bits.
>
> This does mean that you may end up with 150 modules rather than 70.
> Suddenly your processes which were OK at 70 modules are really not
> working with 150, so you may need to be willing to adjust your
> processes etc as well.
+1
More robust structure
Easier to assure that small functional modules are working correctly
Easier to test libraries than applications - usually.
Increases likeliness of sharing of code.

Lots of other good reasons to do this.

Maven and Nexus makes it easy to manage this.

Ron
> Wayne
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org

Re: Best practice for migrating existing system

[Wayne Fay <wa...@gmail.com>]

> As far as I can tell, Option 1 is only practical once the velocity of
> change slows down to the point where at least some components sit
> unchanged for a good long time -- that is, for more than one
> development iteration. It's only useful to 'use a version with a

But it seems like a good (best?) practice to break modules up into
functional components such that parts which have no reason to change
for a given release are not changed. If you find it annoying that a
small change in a small piece of code that is wrapped up inside a much
larger library means the entire library has to bump a version, you may
want to consider breaking the smaller bits out into its own
independent module. Especially if the velocity of change in those
small bits is significantly greater than the velocity of the rest of
the bits.

This does mean that you may end up with 150 modules rather than 70.
Suddenly your processes which were OK at 70 modules are really not
working with 150, so you may need to be willing to adjust your
processes etc as well.

Wayne

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org

Re: Best practice for migrating existing system

[Benson Margulies <bi...@gmail.com>]

Having tried more or less Ron's prescription, I have to offer a
caveat. You spend a lot of time watching the release plugin running.

Frequently, a group of inter-related components has some component
that accumulates shared utilities. In my experience, at least, the
shared component is very busy -- all the users of it find reasons to
change it or add things to it. Call it 'com.mon:common' (groupid
com.mon, artifactid common).

So, you start out on day 1 with all versions at 1-SNAPSHOT. To try to
take Ron's prescription seriously, the first thing you do is run the
release plugin on everything. OK, now there's a '1' of each component
in your copy of nexus. And now you have to make a decision.

Option 1: All the other projects reference com.mon:common:1. Everyone
is using the version with the warranty: for the five minutes that
elapse before everyone feels the need to make a modification to
com.mon:common. Now, each of those people grimaces, and modifies the
pom of the thing they are working on to say, '2-SNAPSHOT' for the
dependency on com.mon:common. When it all works, they check in their
new version of the source of com.mon:common. Now they have to release
it, and then update everything to use the new version. And probably
collide with other people who just did the same thing.

Option 2: All the other projects reference com.mon:common:2-SNAPSHOT.
To maximize sanity, assume that *no-one ever deploys a snapshot*.
Everyone downloads all the source and builds everything, so that they
have a local snapshot. They make whatever changes they need to, and
check in as needed. Once per iteration (scrum-speaking), someone does
an organized job of releasing everything.

As far as I can tell, Option 1 is only practical once the velocity of
change slows down to the point where at least some components sit
unchanged for a good long time -- that is, for more than one
development iteration. It's only useful to 'use a version with a
warranty' if that you can use that version, unchanged, for your
iteration. If you have to make changes, or use the latest changes in
svn, every day, you might as well just be building it yourself and use
your local SNAPSHOT.

Note that 'automatically releasing and bumping version numbers every
night' is here nowhere. It leads to constant churn in the pom files,
source control merge conflicts, and other distress.

Releasing something, *in my opinion* (and all of this is a matter of
*opinion*) only makes sense when that release will have a half-life of
at least a week.

Every 'release' results in changes to all the poms, plus tagging and
bagging in svn.

If you want to test packages every night, my suggestion is to test a
snapshot, and use the build number plugin to assign a number that is
both unique and suitable concise. Further, *do not* publish those
snapshots to a repo where developers will download them. That leads to
the dreaded 'SNAPSHOT skew screw' and you don't want that. Have the
build number plugin capture the svn rev # for the checkout of the
tree. That's all you need to be able to repro any test failures, you
don't need an svn tag.

If the pitfalls and evils of deploying SNAPSHOTs aren't clear to any
interested reader, I'd be happy to type up the standard explanation.

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org

Re: Best practice for migrating existing system

[Ron Wheeler <rw...@artifact-software.com>]

On 07/05/2011 12:10 PM, David McCabe wrote:
> We've been using a custom-made component-based build system for many years,
> built using a number of Ant scripts, and we'd really like to move to
> something more standard. We'd like to go with Maven, Subversion and Nexus,
> but our build processes are a little different from the usual and it would
> be nice to know what is the 'right' way to address some of the versioning
> issues.
>
> Our applications are made up of a few dozen components, some of which are
> shared between projects and some of which are project-specific. Each of
> these components currently has a 4-component build number
> "major.minor.patch.build". When building a component, its build number is
> incremented. A new major, minor or patch version resets the build version to
> zero. I'm happy to change a dot to a hyphen, and I'd also be perfectly happy
> for that build number to be a Subversion revision number, but I'm not happy
> to go for a SNAPSHOT-style date/time: try telling someone your latest
> version is "3.6.0.20110507170144".
A SNAPSHOT is not a release and ideally should never be spoken off 
outside the immediate team and even then you need to explain what a 
SNAPSHOT does and does not yet do and what are the restrictions on its 
use and where it does not respect the specifications. Or you can just 
let the other team members curse you out when they spend hours trying to 
find out why their software does not work when they use your SNAPSHOTs.

SNAPSHOTS are very useful but version 1.3.5.6 is not the same as 
1.3.5.6-SNAPSHOT.
One comes with a warranty and the other comes with a warning.


> The 'released' version of a component is the last build number that
> component is built with. Once a component has been 'released' it cannot be
> built as a new version; the rest of the version number needs to be
> incremented. This is somewhat like Maven's release process except that we do
> not want to renumber or rebuild the components for release; integration and
> system tests will reveal when a project (and its constituent components) are
> ready for release, and we're not going to re-run the entire test suite on a
> functionally identical build. It would be nice to protect a 'released'
> component from being rebuilt but that's hardly mandatory.
>
Nexus will prevent you from redeploying the same release so you have 
some protection. In Maven, releases are immutable.
If you follow best practices with your POM and stay away from such evils 
as "Version Ranges", your builds are repeatable.
Never build a release with SNAPSHOT - there lies the road to damnation!

> Obviously, when a component is built and its build number incremented, all
> dependencies need to have their version numbers updated and be rebuilt. It
> would be nice if the dependencies could automatically pick up the 'latest'
> version of a major.minor.patch dependency version; I understand this can be
> done using Maven version ranges.
>
Bad idea.
Leads to unrepeatable builds.
Make people release dependencies with a warranty.
It is better to use SNAPSHOTs until you are ready to release the top 
level modules and insist that the lower level dependencies be released 
first.
Everyone keeps their versions at SNAPSHOTs until they warrant it is 
release quality.
> When doing a 'real' build I would like the components to be 'tagged' in
> Subversion with the built version numbers and for that tag to reference the
> actual set of build dependencies (rather than a version range). Ideally, I'd
> have a Subversion style:
>
> Component-name
> |- trunk
> |- branches
> |- |- 1.0.0
> |- |- 1.0.1
> |- |- 1.1.0
> |- tags
> |- |- 1.0.0-0
> |- |- 1.0.0-1
> |- |- 1.0.1-0
>
Yes. You are right. You can tag SNAPSHOTS but people seldom do.

> I would also like to be able to do 'test' builds without tagging; maybe the
> tagging should be separate so I can try to build the whole lot to my local
> repository, and if the build is successful and all automated tests pass
> along the way only then update Nexus and tag in Subversion.
Nexus will handle SNAPSHOTS and you can deploy them as long as your 
friends are made aware of where you are in terms of functionality and 
robustness.
> What I want, therefore, is a system in which, for a set of top-level
> components to be built, it would then evaluate the set of transitive
> dependencies of those components as specified in Subversion (not Nexus or
> the local repository), work out the complete list of out-of-date components
> to build and then build all of those components such that the top-level
> components represent the current state of Subversion. Plainly, if the build
> is up-to-date with Subversion, nothing should be built.
>
Too much work. Just run a clean development workflow and do things the 
Maven way and your life will be much more pleasant and your development 
team much happier with each other.
Guys who release crap will soon find out that their friends will jump on 
them and the boss will know that a crappy piece of software was released 
when it was still at the SNAPSHOT stage.

> I accept that this is a job for Python, but I am wondering how much of this
> process can be done with Maven without cluttering the per-project POMs, even
> if minor modifications need to be made.
>
Try not to reinvent a process that works for thousands of other projects.

> Thanks in advance,
> David McCabe
>

Our LMS Portal application has over 70 Maven projects and uses well over 
50 third party libraries. It has been in production and development for 
more than 3 years.
It includes Tomcat servlets, Web Services and standalone batch jobs.
We use a very clean out-of-the-box Maven workflow with no odd plug-ins 
or version ranges.
Maven and Nexus are a great combination. We also use Subversion.

Good luck and you will find this forum a great place for advice. Not 
everyone is as opinionated as me and some of them are really great Maven 
experts and are willing to help others a lot.
You did a very good thing by starting off with a description of what you 
are trying to accomplish. A lot of people show up with detailed 
questions with no context and get "right" answers that lead them down a 
bad path that takes weeks to straighten out.

I hope that this helps.

Ron
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org