You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@maven.apache.org by Giovanni Azua <br...@swissonline.ch> on 2008/05/25 13:05:24 UTC
conflicts resolution and use of md5sum
Hi,
I have tried researching these questions but I did not find any
authoritative answer:
1) If via transitive dependency resolution there is a conflict found i.e.
two dependencies use different versions of a fourth library, how does maven
resolves this? This is a.k.a. diamond problem:
"My Project"
| |-----> A ---> C (version 1.0)
|
|------------> B ---> C (version 1.1)
If both C versions are pulled there will be runtime unpredictable behavior.
2) If the point 1) is checked by Maven, how is it done? Using only version
numbers or md5sum? Is it possible enforcing the use of md5sum for this
purpose?
If you know where these questions are answered, pointing me to the location
will be most welcome!
TIA,
Best regards,
Giovanni
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: conflicts resolution and use of md5sum
Posted by Michael McCallum <gh...@apache.org>.
On Mon, 26 May 2008 02:55:23 Arun Kumar wrote:
> my guess is that both jars will be downloaded but the first one will get
> the precedence because of the classpath
Why would you guess? The code is there or you could write some simple test
poms and do mvn -X dependency:resolve to find out what happens...
--
Michael McCallum
Enterprise Engineer
mailto:gholam@apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: conflicts resolution and use of md5sum
Posted by Arun Kumar <pg...@gmail.com>.
my guess is that both jars will be downloaded but the first one will get the
precedence because of the classpath
ideally, in a single project both A and B should use the same version.. why
would we want to happen this?
On Sun, May 25, 2008 at 7:58 PM, Brian E. Fox <br...@reply.infinity.nu>
wrote:
> Maven uses a nearest first, but in the example you provided C is the same
> depth. In this case it will go with the first version of C found, which is
> dependent on if A or B is listed first in your pom.
>
>
> On 5/25/08 7:05 AM, "Giovanni Azua" <br...@swissonline.ch> wrote:
>
> > Hi,
> >
> > I have tried researching these questions but I did not find any
> > authoritative answer:
> >
> > 1) If via transitive dependency resolution there is a conflict found i.e.
> > two dependencies use different versions of a fourth library, how does
> maven
> > resolves this? This is a.k.a. diamond problem:
> >
> > "My Project"
> > | |-----> A ---> C (version 1.0)
> > |
> > |------------> B ---> C (version 1.1)
> >
> > If both C versions are pulled there will be runtime unpredictable
> behavior.
> >
> > 2) If the point 1) is checked by Maven, how is it done? Using only
> version
> > numbers or md5sum? Is it possible enforcing the use of md5sum for this
> > purpose?
> >
> > If you know where these questions are answered, pointing me to the
> location
> > will be most welcome!
> >
> > TIA,
> > Best regards,
> > Giovanni
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> > For additional commands, e-mail: users-help@maven.apache.org
> >
> >
>
>
--
cheers,
- a
Re: conflicts resolution and use of md5sum
Posted by "simon.kitching@chello.at" <si...@chello.at>.
Michael McCallum schrieb:
> rubbish, that is exactly what ranges and release poms are for, although watch
> out for missing exclusions in release poms...
>
> On Mon, 26 May 2008 05:41:51 Brian E. Fox wrote:
>
>>> new different version each time. AFAIK there is nothing in Maven that
>>> prevents from re-releasing the same version number/label several times,
>>> therefore it is a Pandora Box against reproducibility of builds.
>>>
>> You simply cannot re-release the same release version of an artifact and
>> expect Maven to work correctly. This is what SNAPSHOTS are for.
>>
I say double rubbish :-).
There are two *very* good reasons to not re-release a (non-snapshot)
artifact with the same version number.
Firstly, when maven sees that a project it is building needs version
1.2.3 of an artifact, and the local repository alreads holds that
version, then it will *not* check any remote repository. Therefore if
people re-release artifacts with the same version number, then builds on
two different machines can behave differently, depending on what they
happened to have cached locally. Very bad. Note that SNAPSHOTS have an
"update time" (every-time, daily, never) which controls how often a real
repo is checked, but that doesn't exist for non-snapshot artifacts.
Secondly, even if maven did check the repo every time, overwriting an
artifact with a new one still destroys *all* chances of repeatable
builds. The version that existed yesterday would be gone without hope of
retrieval, so the build done yesterday could not be repeated with the
same result. Very very bad.
So as Brian says: "You simply cannot re-release the same version of an
artifact".
PS: http://www.nabble.com/How-to-reply-to-emails-t3182877.html
Regards,
Simon
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: conflicts resolution and use of md5sum
Posted by Michael McCallum <gh...@apache.org>.
rubbish, that is exactly what ranges and release poms are for, although watch
out for missing exclusions in release poms...
On Mon, 26 May 2008 05:41:51 Brian E. Fox wrote:
> >new different version each time. AFAIK there is nothing in Maven that
> >prevents from re-releasing the same version number/label several times,
> >therefore it is a Pandora Box against reproducibility of builds.
>
> You simply cannot re-release the same release version of an artifact and
> expect Maven to work correctly. This is what SNAPSHOTS are for.
--
Michael McCallum
Enterprise Engineer
mailto:gholam@apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
RE: conflicts resolution and use of md5sum
Posted by simon <si...@chello.at>.
On Sun, 2008-05-25 at 18:28 +0200, Giovanni Azua wrote:
> Hi Simon,
>
> Many thanks to you all for the replies!
>
> The issue here is reliability and reproducibility of the builds.
>
> 1) Locking down versions is needed for reproducibility of the builds. If
> maven decides silently for you what the resolution of conflicts is going to
> be, most likely you will end up with a runtime time bomb just waiting to
> blow. Very likely one of the two dependencies (A or B) will fail with
> whatever the C selected version C is. IMO the most sensitive thing to do in
> this scenario would be to have maven breaking the build explaining the
> conflict and letting the user fix the dependencies. If maven silently
> decides, the only provision developers have against runtime troubles is
> having high test coverage in place, assuming high test coverage is usually
> wishful thinking.
Building of poms is already deterministic, as long as no dependency has
a SNAPSHOT version [1]. It is an error in maven if a dependency is
specified without a version. The version can be a "recommendation" type
version, ie no range specified, but even with this it is still 100%
deterministic which version you get.
Having poms for library projects lock down their dependencies to an
exact version would be a nightmare. It would soon be completely
impossible to assemble any set of jars which have compatible
dependencies.
By your rule, any lib that declares a dependency on commons-logging 1.1
would be unusable with any lib that declares a dependency on
commons-logging-1.1.1. But they *are* compatible.
When you start a new project, ie create a pom with a set of direct
dependencies, Maven's transient dependency resolution
*deterministically* pulls together a set of libs that have a pretty good
chance of being what are needed. It is possible that there are some
incompatibilities in there, but the top-level pom can then tweak things
as needed. As pointed out in earlier replies, a dependency declaration
with a *lower depth* overrides others, so declarations in the top-level
pom override all transient declarations (except ones with ranges). And
"mvn dependency:tree" shows exactly what is going on.
A decent set of unit tests for a project *is* the correct way to handle
this situation, verifying that the selected libs do function together
correctly.
>
> 2) Using md5sum is a safer way to ensure that the same specific release is
> being pulled e.g. an scenario like the one explained in the OP. Using md5sum
> also favors reproducibility, and it is specially useful in places where the
> same release version of a component is re-released several times (I work in
> one of those places) whereas in maven it implies you diligently provide a
> new different version each time. AFAIK there is nothing in Maven that
> prevents from re-releasing the same version number/label several times,
> therefore it is a Pandora Box against reproducibility of builds.
The main maven repository has a policy of never overwriting a release.
When version X of a library is installed into the central maven
repository, you can 100% rely on that never changing.
The only exception is a SNAPSHOT release. But those don't promise
repeatable builds.
It is strongly recommended that you follow the same policy for your own
repositories. Don't re-release artifacts with the same version number
(except for SNAPSHOTs). You can probably enforce this by setting the
appropriate UMASK for your repository, ie making sure that files in the
repository do not have write permission.
I suppose it might be nice to be able to write
<dependency>
<groupId>example</groupId>
<artifactId>widget</artifactId>
<version>[1.2]</version>
<md5sum>DEADBEEF</md5sum>
</dependency>
and have an error if the downloaded file doesn't match the checksum.
That would indeed be an absolute check that the file is the expected
one. But it only works for "locked versions" such as [1.2], and
therefore is of no use in poms for libraries of any sort. I suppose
maven could offer a tool that works like
mvn dependency:tree
but outputs the checksums. Then that could be pasted into the top-level
pom.xml file as a real paranoia check. But I doubt many people would
take advantage of that.
If you are in a really secure environment where you need to be sure that
the jars in the repo are not being modified, then you need to run your
own repository instead. Use a "repository manager" to pull down the
files your project needs into a temporary "unsecure" location, then move
them to the secure repository, validating them in whatever manner you
need. Then all builds can be run against your own secure, guaranteed
stable repository.
[1] And this is easy enough to check, just by running "mvn
dependency:list", or using the maven-release-plugin.
Regards,
Simon
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: conflicts resolution and use of md5sum
Posted by Stephen Connolly <st...@gmail.com>.
On Sun, May 25, 2008 at 5:28 PM, Giovanni Azua <br...@swissonline.ch>
wrote:
> Hi Simon,
>
> Many thanks to you all for the replies!
>
> The issue here is reliability and reproducibility of the builds.
>
> 1) Locking down versions is needed for reproducibility of the builds. If
> maven decides silently for you what the resolution of conflicts is going to
> be, most likely you will end up with a runtime time bomb just waiting to
> blow. Very likely one of the two dependencies (A or B) will fail with
> whatever the C selected version C is. IMO the most sensitive thing to do in
> this scenario would be to have maven breaking the build explaining the
> conflict and letting the user fix the dependencies. If maven silently
> decides, the only provision developers have against runtime troubles is
> having high test coverage in place, assuming high test coverage is usually
> wishful thinking.
>
That won't work if the source of your conflict is a third party dependency.
If you need to lock down, use version ranges
>
> 2) Using md5sum is a safer way to ensure that the same specific release is
> being pulled e.g. an scenario like the one explained in the OP. Using
> md5sum
> also favors reproducibility, and it is specially useful in places where the
> same release version of a component is re-released several times (I work in
> one of those places) whereas in maven it implies you diligently provide a
> new different version each time. AFAIK there is nothing in Maven that
> prevents from re-releasing the same version number/label several times,
> therefore it is a Pandora Box against reproducibility of builds.
>
md5sum is not easy to type. version is and it is reproducible
>
> Many thanks!
>
> Best regards,
> Giovanni
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
>
RE: conflicts resolution and use of md5sum
Posted by "Brian E. Fox" <br...@reply.infinity.nu>.
>Using md5sum
>also favors reproducibility, and it is specially useful in places where
the
>same release version of a component is re-released several times (I
work in
>one of those places) whereas in maven it implies you diligently provide
a
>new different version each time. AFAIK there is nothing in Maven that
>prevents from re-releasing the same version number/label several times,
>therefore it is a Pandora Box against reproducibility of builds.
You simply cannot re-release the same release version of an artifact and
expect Maven to work correctly. This is what SNAPSHOTS are for.
--Brian
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
RE: conflicts resolution and use of md5sum
Posted by Giovanni Azua <br...@swissonline.ch>.
Hi Simon,
Many thanks to you all for the replies!
The issue here is reliability and reproducibility of the builds.
1) Locking down versions is needed for reproducibility of the builds. If
maven decides silently for you what the resolution of conflicts is going to
be, most likely you will end up with a runtime time bomb just waiting to
blow. Very likely one of the two dependencies (A or B) will fail with
whatever the C selected version C is. IMO the most sensitive thing to do in
this scenario would be to have maven breaking the build explaining the
conflict and letting the user fix the dependencies. If maven silently
decides, the only provision developers have against runtime troubles is
having high test coverage in place, assuming high test coverage is usually
wishful thinking.
2) Using md5sum is a safer way to ensure that the same specific release is
being pulled e.g. an scenario like the one explained in the OP. Using md5sum
also favors reproducibility, and it is specially useful in places where the
same release version of a component is re-released several times (I work in
one of those places) whereas in maven it implies you diligently provide a
new different version each time. AFAIK there is nothing in Maven that
prevents from re-releasing the same version number/label several times,
therefore it is a Pandora Box against reproducibility of builds.
Many thanks!
Best regards,
Giovanni
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: conflicts resolution and use of md5sum
Posted by simon <si...@chello.at>.
But if version ranges are specified anywhere, then they take priority.
A version declaration without a range is effectively a "recommendation".
In your example, there are two "recommendations", so (as Brian said) the
one with the least "depth" is used, and in case of a tie then the first
found is used.
But if B declares a dependency on C version "[1.1]" then version 1.1 is
what will be used. Note that explicitly locked-down versions like this
are not generally a good idea in reusable libs, as it makes the lib
incompatible with anything declaring a dependency on [1.2,) for example;
when two conflicting ranges exist then maven will declare that there is
no available version and stop.
In no case will two copies of C be placed in the classpath. And the
behaviour is completely deterministic.
I'm sure this is documented in one or more of the Maven books available
online.
I'm not sure what your question about checksums was about. Maven never
needs to figure out what version a specific jarfile is. It first
determines what version it should pull in, then looks in the appropriate
place in a repository. Checksums exist only to detect data corruption
during upload or download of files.
Regards,
Simon
On Sun, 2008-05-25 at 10:28 -0400, Brian E. Fox wrote:
> Maven uses a nearest first, but in the example you provided C is the same
> depth. In this case it will go with the first version of C found, which is
> dependent on if A or B is listed first in your pom.
>
>
> On 5/25/08 7:05 AM, "Giovanni Azua" <br...@swissonline.ch> wrote:
>
> > Hi,
> >
> > I have tried researching these questions but I did not find any
> > authoritative answer:
> >
> > 1) If via transitive dependency resolution there is a conflict found i.e.
> > two dependencies use different versions of a fourth library, how does maven
> > resolves this? This is a.k.a. diamond problem:
> >
> > "My Project"
> > | |-----> A ---> C (version 1.0)
> > |
> > |------------> B ---> C (version 1.1)
> >
> > If both C versions are pulled there will be runtime unpredictable behavior.
> >
> > 2) If the point 1) is checked by Maven, how is it done? Using only version
> > numbers or md5sum? Is it possible enforcing the use of md5sum for this
> > purpose?
> >
> > If you know where these questions are answered, pointing me to the location
> > will be most welcome!
> >
> > TIA,
> > Best regards,
> > Giovanni
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> > For additional commands, e-mail: users-help@maven.apache.org
> >
> >
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: conflicts resolution and use of md5sum
Posted by "Brian E. Fox" <br...@reply.infinity.nu>.
Maven uses a nearest first, but in the example you provided C is the same
depth. In this case it will go with the first version of C found, which is
dependent on if A or B is listed first in your pom.
On 5/25/08 7:05 AM, "Giovanni Azua" <br...@swissonline.ch> wrote:
> Hi,
>
> I have tried researching these questions but I did not find any
> authoritative answer:
>
> 1) If via transitive dependency resolution there is a conflict found i.e.
> two dependencies use different versions of a fourth library, how does maven
> resolves this? This is a.k.a. diamond problem:
>
> "My Project"
> | |-----> A ---> C (version 1.0)
> |
> |------------> B ---> C (version 1.1)
>
> If both C versions are pulled there will be runtime unpredictable behavior.
>
> 2) If the point 1) is checked by Maven, how is it done? Using only version
> numbers or md5sum? Is it possible enforcing the use of md5sum for this
> purpose?
>
> If you know where these questions are answered, pointing me to the location
> will be most welcome!
>
> TIA,
> Best regards,
> Giovanni
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
>