You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@maven.apache.org by Keith Branton <kb...@overstock.com> on 2008/10/04 03:16:57 UTC
Version conflict resolution and stable builds
I've been spending some time trying to get my head around the way
version resolution works in 2.0.9.
I'm trying to get a sense for the best way to use maven to achieve our
objectives:
1. Builds on our CI server that were working should not break unless
someone commits a change - in particular new versions of dependencies
should not be introduced to the build without a change to the pom. (I'm
actually more concerned about changes that don't break the build on CI
but cause runtime errors on our production web site due to some edge
case that was insufficiently covered with automated tests.)
2. We really don't want to have to restate the versions of every
dependency in all local pom files. We like "dry".
I've looked at the version ranges but am a little concerned about them
for a couple of reasons:
1. Closed ranges like [1.2,1.4] means modules need to be re-tested and
re-released with a new range if a dependency changes and any application
using the module needs the newer version of the dependency, say 1.5,
even if only transitively. With the hundreds of modules we have, this
would be a time-consuming task. Besides, very few 3rd party modules seem
to use ranges.
2. Open ranges like [1.2,) seem to imply a guarantee of compatibility
with ALL future releases. (And if all dependency declarations for a
module A are open ranges then the version of A that is actually included
can change over time as maven pulls in the latest one from the repo -
leading to temporally inconsistent builds)
I studied MNG-612 which seems like it goes a little way towards
addressing our issue, but the status of that ticket is unclear as it's
pretty old and none of the code has been merged back. Any info welcome.
(The MNG-612 branch looks like it introduced a bug into the scope
resolving code too as it swaps nearest and farthest under some
circumstances - and it is not clear to me that it would work with
ranges.)
I think what *we* would want would be the following rules for version
conflict resolution...
If a suggested version is specified in the local pom
use it. Treat it as override for transitives, unless any
transitive specifies a range that is not satisfied by the suggestion -
in which case resolution should fail.
Else If any ranges are used
select the oldest version that satisfies all the ranges.
Else
use the newest suggested version.
The scope rules would need to stay as they are in 2.0.9
My questions:
1. Can anyone please tell me of a way to achieve this with 2.0.9 today?
2. Can 2.1 do this yet? Or is support for this functionality planned for
any 2.1 milestone?
3. Can anyone point me to discussions/documents describing what
direction the maven developers are planning for version conflict
resolution in the future?
If this is not in the works and we could contribute to maven to help
make this happen we would be willing to do so.
Thanks,
Keith.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Version conflict resolution and stable builds
Posted by Michael McCallum <gh...@apache.org>.
On Fri, 10 Oct 2008 03:00:38 Zoltan Farkas wrote:
> I believe using dependencyManagement actually helps you... it gives you
> the power to control the dependency versions...
in very simple cases it helps and with the import scope it can be quite useful
but its more difficult to understand the result of depedency resolution.
The issue I have is that depedency management is an attempt to force a version
of a library but from outside of the resolution tree. It has side effects and
ultimately it makes it harder to manage dependencies because it makes it easy
consolidate all your version for a short term gain.
For a small project it makes sense and works. For larger projects it does not.
And I take the java view, fewer more well understood approaches to problems
leads to better code sometimes more verbose but easy to understand.
But even more than that when I manage versions of external libraries I just
just manage versions, I also manage related libraries and exclusions and
probably the most important: I provide an independent "release cycle" from
the upstream projects which means all my projects can be exposed in stages to
upstream releases.
I fear that most of the things I say will make no sense to windows and linux
users and a reaonsable portion of linux users who don't really understand how
far package management has come and its parallels with maven.
>
> from the doc:
>
> "very important use of the dependency management section is to control
> the versions of artifacts used in transitive dependencies"
>
> --zoly
--
Michael McCallum
Enterprise Engineer
mailto:gholam@apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Version conflict resolution and stable builds
Posted by Michael McCallum <gh...@apache.org>.
On Thu, 09 Oct 2008 22:31:10 Michael McCallum wrote:
> theres a taste i probably missed something as this is just off the top of
> my head,
yes i did
don't use release poms if you expect version ranges to work... its just makes
a mess of everything and gets rid of your carefully crafted exclusions...
this might have been fixed i'd have to go and test... i should have opened a
bug if there is not one but just worked around it for the time being
--
Michael McCallum
Enterprise Engineer
mailto:gholam@apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Version conflict resolution and stable builds
Posted by Zoltan Farkas <zo...@daxtechnologies.com>.
I believe using dependencyManagement actually helps you... it gives you
the power to control the dependency versions...
from the doc:
"very important use of the dependency management section is to control
the versions of artifacts used in transitive dependencies"
--zoly
Michael McCallum wrote:
> Lets got back to the basics of development... you need to develop something to
> that is delivered to be run....
>
> So by my way of thinking I want an acurate description of whats delivered and
> run such that I can reproduce it. Have the convention that versions are in
> the jar names and the pom and its properties end up in the jar facilitates
> this.
>
> But I also need to have a flexible develoment environment with low overheads.
> If I use best guess or open ranges I have no power to control things because
> i need to tweak poms all over the place.
>
> So my approach is a design and build by contract when i use the major version
> to indicate a contract... and all my ranges restrict a dependency to a single
> major version. e.g.
> a.b.c-1.5 depends on a.b.d with [1,2)
> This is arbitrarily simple see a more complex example attached
>
> Now what this means is that when you break your project down into its
> component pieces you can by convention relate them to other things stably.
> Not reproducibly but thats ok because its determinable. If you try to build
> your deliverable and there is a range conflict you get an error on build...
> you can't build a dodgy deliverable.
> With open version ranges or best guess deps you can end up with any released
> dependency! and even if you have a process to know when you got it wrong but
> its very difficult other than adding dependencies to the deliverable project
> to get it right.
>
> Now your developers need to make a call when they release - usually after
> discussion - is my change breaking - you can test this with CI - and if so
> should I increment the major version. If you get broken because they make a
> mistake you have options... roll back deps... release a new rolled back
> version...
>
> The funny thing I avoided most of the gotches due to my approach...
> Now the gotchas...
> 1) don't use modules
> 2) don't use snapshot parents or relative paths
> 3) inherit by function which means have parents that configure particular
> plugins and keep them simple and try to have a little as possible in children
> poms. i have a 2 deep hierarhcy with one root and 9 sub parents
> The convention i use to break projects down is by function that means I have
> api, model aka domain, strategy, composition, configuration, webapp, abstract
> webapp, webservice, repository. It means a proliferation of artifacts but at
> the same time a proliferation of reuse...
> 4) use a 2 point version major.release and always start at X.1 never at X.0
> otherwise ranges don't work... its not a bug just an ideosyncracy... consider
> [1,2) this does not match 1.0-SNAPSHOT and matches 2.0-SNAPSHOT which seems
> odd... if you never make 1.0-SNAPSHOT or 2.0-SNAPSHOT then range boundaries
> work determinably. range calculations break down if have more that 3 points
> so you need to leave wiggle room.
> 5) save the third point for patches which are releases off branches
> 6) don't use dependency management it encourages bad inheritance, you can just
> use transitive dependencies and they are more flexible
> 7) wrap 3rd party dependencies with local projects that just enforce
> consistent resolution - they don't use ranges so you need to manage it
>
> theres a taste i probably missed something as this is just off the top of my
> head,
>
> gotta do some work now
>
>
> On Thu, 09 Oct 2008 12:47:13 Ian Robertson wrote:
>
>> On Sun, 2008-10-05 at 04:48 -0600, Michael McCallum wrote:
>>
>>> On Sat, 04 Oct 2008 14:16:57 Keith Branton wrote:
>>>
>>>> 1. Can anyone please tell me of a way to achieve this with 2.0.9 today?
>>>>
>>> With the appropriate use of ranges you can do this and it resolves as
>>> described. There are about 11 gotchas to doing it though, all worked
>>> around by appropriate conventions.
>>>
>> Can you elaborate on what these 11 gotchas are, or point to a place
>> which does?
>>
>> One of the biggest issues I see with ranges is that maven chooses the
>> most recent version in the repository which matches a given range.
>> Consequently, if a pom specifies a version range for a dependency like
>> [1.5,), the exact same code can create different builds on different
>> days if a new release of the dependency is made.
>>
>> I'm actually curious if anyone knows the reason for this; it seams that
>> a far better choice would be to choose the *lowest* version which
>> matches all range requirements, so that builds could be temporally
>> stable.
>>
>> - Ian
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
>> For additional commands, e-mail: users-help@maven.apache.org
>>
>
>
>
>
> ------------------------------------------------------------------------
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Version conflict resolution and stable builds
Posted by Michael McCallum <gh...@apache.org>.
Lets got back to the basics of development... you need to develop something to
that is delivered to be run....
So by my way of thinking I want an acurate description of whats delivered and
run such that I can reproduce it. Have the convention that versions are in
the jar names and the pom and its properties end up in the jar facilitates
this.
But I also need to have a flexible develoment environment with low overheads.
If I use best guess or open ranges I have no power to control things because
i need to tweak poms all over the place.
So my approach is a design and build by contract when i use the major version
to indicate a contract... and all my ranges restrict a dependency to a single
major version. e.g.
a.b.c-1.5 depends on a.b.d with [1,2)
This is arbitrarily simple see a more complex example attached
Now what this means is that when you break your project down into its
component pieces you can by convention relate them to other things stably.
Not reproducibly but thats ok because its determinable. If you try to build
your deliverable and there is a range conflict you get an error on build...
you can't build a dodgy deliverable.
With open version ranges or best guess deps you can end up with any released
dependency! and even if you have a process to know when you got it wrong but
its very difficult other than adding dependencies to the deliverable project
to get it right.
Now your developers need to make a call when they release - usually after
discussion - is my change breaking - you can test this with CI - and if so
should I increment the major version. If you get broken because they make a
mistake you have options... roll back deps... release a new rolled back
version...
The funny thing I avoided most of the gotches due to my approach...
Now the gotchas...
1) don't use modules
2) don't use snapshot parents or relative paths
3) inherit by function which means have parents that configure particular
plugins and keep them simple and try to have a little as possible in children
poms. i have a 2 deep hierarhcy with one root and 9 sub parents
The convention i use to break projects down is by function that means I have
api, model aka domain, strategy, composition, configuration, webapp, abstract
webapp, webservice, repository. It means a proliferation of artifacts but at
the same time a proliferation of reuse...
4) use a 2 point version major.release and always start at X.1 never at X.0
otherwise ranges don't work... its not a bug just an ideosyncracy... consider
[1,2) this does not match 1.0-SNAPSHOT and matches 2.0-SNAPSHOT which seems
odd... if you never make 1.0-SNAPSHOT or 2.0-SNAPSHOT then range boundaries
work determinably. range calculations break down if have more that 3 points
so you need to leave wiggle room.
5) save the third point for patches which are releases off branches
6) don't use dependency management it encourages bad inheritance, you can just
use transitive dependencies and they are more flexible
7) wrap 3rd party dependencies with local projects that just enforce
consistent resolution - they don't use ranges so you need to manage it
theres a taste i probably missed something as this is just off the top of my
head,
gotta do some work now
On Thu, 09 Oct 2008 12:47:13 Ian Robertson wrote:
> On Sun, 2008-10-05 at 04:48 -0600, Michael McCallum wrote:
> > On Sat, 04 Oct 2008 14:16:57 Keith Branton wrote:
> > > 1. Can anyone please tell me of a way to achieve this with 2.0.9 today?
> >
> > With the appropriate use of ranges you can do this and it resolves as
> > described. There are about 11 gotchas to doing it though, all worked
> > around by appropriate conventions.
>
> Can you elaborate on what these 11 gotchas are, or point to a place
> which does?
>
> One of the biggest issues I see with ranges is that maven chooses the
> most recent version in the repository which matches a given range.
> Consequently, if a pom specifies a version range for a dependency like
> [1.5,), the exact same code can create different builds on different
> days if a new release of the dependency is made.
>
> I'm actually curious if anyone knows the reason for this; it seams that
> a far better choice would be to choose the *lowest* version which
> matches all range requirements, so that builds could be temporally
> stable.
>
> - Ian
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
--
Michael McCallum
Enterprise Engineer
mailto:gholam@apache.org
Re: Version conflict resolution and stable builds
Posted by Michael McCallum <gh...@apache.org>.
On Thu, 09 Oct 2008 12:47:13 Ian Robertson wrote:
> I'm actually curious if anyone knows the reason for this; it seams that
> a far better choice would be to choose the *lowest* version which
> matches all range requirements, so that builds could be temporally
> stable.
ranges would be useless because they would never change, ultimately it assumes
that you are converging on the goal and new releases are better... which is
quite reasonable
--
Michael McCallum
Enterprise Engineer
mailto:gholam@apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Version conflict resolution and stable builds
Posted by Ian Robertson <ir...@overstock.com>.
On Sun, 2008-10-05 at 04:48 -0600, Michael McCallum wrote:
> On Sat, 04 Oct 2008 14:16:57 Keith Branton wrote:
> > 1. Can anyone please tell me of a way to achieve this with 2.0.9 today?
> With the appropriate use of ranges you can do this and it resolves as
> described. There are about 11 gotchas to doing it though, all worked around
> by appropriate conventions.
Can you elaborate on what these 11 gotchas are, or point to a place
which does?
One of the biggest issues I see with ranges is that maven chooses the
most recent version in the repository which matches a given range.
Consequently, if a pom specifies a version range for a dependency like
[1.5,), the exact same code can create different builds on different
days if a new release of the dependency is made.
I'm actually curious if anyone knows the reason for this; it seams that
a far better choice would be to choose the *lowest* version which
matches all range requirements, so that builds could be temporally
stable.
- Ian
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Version conflict resolution and stable builds
Posted by Michael McCallum <gh...@apache.org>.
On Sat, 04 Oct 2008 14:16:57 Keith Branton wrote:
> 1. Can anyone please tell me of a way to achieve this with 2.0.9 today?
With the appropriate use of ranges you can do this and it resolves as
described. There are about 11 gotchas to doing it though, all worked around
by appropriate conventions.
--
Michael McCallum
Enterprise Engineer
mailto:gholam@apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Version conflict resolution and stable builds
Posted by Stephen Connolly <st...@gmail.com>.
have a look at versions-maven-plugin
it may or may not be able to help you
Sent from my iPod
On 4 Oct 2008, at 02:16, "Keith Branton" <kb...@overstock.com> wrote:
> I've been spending some time trying to get my head around the way
> version resolution works in 2.0.9.
>
> I'm trying to get a sense for the best way to use maven to achieve our
> objectives:
>
> 1. Builds on our CI server that were working should not break unless
> someone commits a change - in particular new versions of dependencies
> should not be introduced to the build without a change to the pom.
> (I'm
> actually more concerned about changes that don't break the build on CI
> but cause runtime errors on our production web site due to some edge
> case that was insufficiently covered with automated tests.)
>
> 2. We really don't want to have to restate the versions of every
> dependency in all local pom files. We like "dry".
>
> I've looked at the version ranges but am a little concerned about them
> for a couple of reasons:
>
> 1. Closed ranges like [1.2,1.4] means modules need to be re-tested and
> re-released with a new range if a dependency changes and any
> application
> using the module needs the newer version of the dependency, say 1.5,
> even if only transitively. With the hundreds of modules we have, this
> would be a time-consuming task. Besides, very few 3rd party modules
> seem
> to use ranges.
>
> 2. Open ranges like [1.2,) seem to imply a guarantee of compatibility
> with ALL future releases. (And if all dependency declarations for a
> module A are open ranges then the version of A that is actually
> included
> can change over time as maven pulls in the latest one from the repo -
> leading to temporally inconsistent builds)
>
> I studied MNG-612 which seems like it goes a little way towards
> addressing our issue, but the status of that ticket is unclear as it's
> pretty old and none of the code has been merged back. Any info
> welcome.
> (The MNG-612 branch looks like it introduced a bug into the scope
> resolving code too as it swaps nearest and farthest under some
> circumstances - and it is not clear to me that it would work with
> ranges.)
>
> I think what *we* would want would be the following rules for version
> conflict resolution...
>
> If a suggested version is specified in the local pom
> use it. Treat it as override for transitives, unless any
> transitive specifies a range that is not satisfied by the suggestion -
> in which case resolution should fail.
> Else If any ranges are used
> select the oldest version that satisfies all the ranges.
> Else
> use the newest suggested version.
>
> The scope rules would need to stay as they are in 2.0.9
>
> My questions:
>
> 1. Can anyone please tell me of a way to achieve this with 2.0.9
> today?
>
> 2. Can 2.1 do this yet? Or is support for this functionality planned
> for
> any 2.1 milestone?
>
> 3. Can anyone point me to discussions/documents describing what
> direction the maven developers are planning for version conflict
> resolution in the future?
>
> If this is not in the works and we could contribute to maven to help
> make this happen we would be willing to do so.
>
> Thanks,
>
> Keith.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org