Maven adjusts transitive compile-scope dependency to test-scope

[Simon Taddiken <si...@gmail.com>]

Hi everyone,

I've encountered the following behavior and I'm not quite sure whether it
is desirable.
In my project, I have declared a dependency *X* with scope *test*. I then
updated the version of a 3rd party dependency *Y*. In its new version, *Y*
suddenly requires the aforementioned dependency *X* as a *compile *scoped
dependency.

In this scenario, maven resolves the scope of *X* to be *test, *although it
is now required by *Y* during runtime, causing ClassNotFoundExceptions. By
the very nature of this behavior those mistakes can't even be detected in
unit tests because *X* is available on test classpath.

I have just found out that this behavior seems to be intended (
https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Dependency_Scope)
but I can't really come up with a rationale for this design. Managing any
transitive compile scoped dependency down to test scope will almost
certainly cause ClassNotFoundExceptions during runtime.

Thoughts?

Re: Maven adjusts transitive compile-scope dependency to test-scope

[Jason Young <ja...@procentive.com>]

I think I understand now. You're saying MyProject requires X for testing
and Y for compiling, but Y also requires X for compiling, thus X is a
transitive compile-time (or at least runtime) dependency of MyProject,
incidentally (not directly). Maven is not interpreting your input that way.

You mean to tell Maven that MyProject requires X _in addition to_ other
dependencies specified, but Maven interprets that as _overriding_ or
_replacing_ the transitive dependency. If I understand correctly, your
original post was saying that seems worng or at least needlessly opens the
door to mistakes which a different design could prevent. E.g. if Maven
required you to use an exclusion to remove a dependency instead of allowing
you to quietly override a dependency with a weaker scope, then you could
_just_ specify what a project needs directly, and Maven would figure out
the details or fail the build if what you ask for has an ambiguity with no
safe, meaningful default resolution. Hope that makes sense.

On Wed, May 15, 2019 at 12:42 AM Simon Taddiken <si...@gmail.com>
wrote:

> In my case, the pom of Y explicitly declares X with scope compile. The
> dependency tree should look something like this:
>
> MyProject <- this is a standalone application, that will not be depended on
> by s/o else
> |- X:test
> |- Y:compile
>    \- X:compile
>
> So for MyProject, X is explicitly declared test but X also comes in
> transitively via Y as compile. I do understand that declarations higher in
> the tree are given precedence when it comes to dependency resolution but in
> this case it simply breaks runtime code.
> Of course I did not want to ship MyProject including X when I used X only
> for tests. But now that Y also requires X I need MyProject to ship with X
> in order to work correctly.
>
> I was just wondering whether there is some mechanism like a warning or a
> flag to break the build in such cases, so that they do not go unnoticed.
>
>
> Am Di., 14. Mai 2019 um 22:22 Uhr schrieb Jason Young <
> jason.young@procentive.com>:
>
> > Did you declare that Y depends on X at all (via Y's pom.xml) or did it
> > figure that out on its own (via transitive dependencies)?
> >
> > Test scope means it's for testing that one project, thus it's not
> > transitive, e.g. I don't need JUnit for my tests just because
> > SomeAwesomeProject uses JUnit for its tests, and I don't want to ship
> JUnit
> > in my project.
> >
> > On Tue, May 14, 2019 at 2:40 PM Simon Taddiken <simon.taddiken@gmail.com
> >
> > wrote:
> >
> > > Hi everyone,
> > >
> > > I've encountered the following behavior and I'm not quite sure whether
> it
> > > is desirable.
> > > In my project, I have declared a dependency *X* with scope *test*. I
> then
> > > updated the version of a 3rd party dependency *Y*. In its new version,
> > *Y*
> > > suddenly requires the aforementioned dependency *X* as a *compile
> *scoped
> > > dependency.
> > >
> > > In this scenario, maven resolves the scope of *X* to be *test,
> *although
> > it
> > > is now required by *Y* during runtime, causing ClassNotFoundExceptions.
> > By
> > > the very nature of this behavior those mistakes can't even be detected
> in
> > > unit tests because *X* is available on test classpath.
> > >
> > > I have just found out that this behavior seems to be intended (
> > >
> > >
> >
> https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Dependency_Scope
> > > )
> > > but I can't really come up with a rationale for this design. Managing
> any
> > > transitive compile scoped dependency down to test scope will almost
> > > certainly cause ClassNotFoundExceptions during runtime.
> > >
> > > Thoughts?
> > >
> >

Re: Maven adjusts transitive compile-scope dependency to test-scope

[Andy Feldman <an...@wealthfront.com>]

I agree that this behavior seems risky. It's especially bad that the issue
cannot be caught in unit tests, even those which cover the code paths in Y
that call X.

I don't think the idea of manually checking the dependency tree is
realistic for larger projects, but the enforcer plugin seems perfect. I
don't see a built-in rule that could catch this. The documentation claims
it would be easy to write:
https://maven.apache.org/enforcer/enforcer-api/writing-a-custom-rule.html.
It would be like RequireUpperBoundDeps, except instead of comparing version
numbers, it would compare scope. Here's the source for that check if
someone wanted to use it as a starting point:
https://maven.apache.org/enforcer/apidocs/src-html/org/apache/maven/plugins/enforcer/RequireUpperBoundDeps.html
.

Having worked on a fairly large project for a few years, my team hasn't
been bitten by this issue, so I probably won't write this myself. It's
usually pretty consistent which dependencies are for test scope (e.g.
JUnit) and which are not (e.g. Guava), so it just hasn't come up that an
upstream library has used a different scope than we have. But if someone
were to implement this plugin I would include it in our projects just in
case.


On Tue, May 14, 2019 at 10:51 PM Thomas Broyer <t....@gmail.com> wrote:

> There might be a rule in the enforcer plugin (I haven't checked). If one
> really wanted to change the dependency scope, he could add an exclusion in
> the Y dependency to break the transitivity.
>
> Fwiw, whenever you change your dependencies, it's a good idea to run mvn
> dependency:tree and look at the output (possibly diff'ing it with the
> output prior to the dependency change).
> Maven basically assumes that you take responsibility for the dependencies,
> and it only takes the role of a helper resolving the whole tree.
>
> Le mer. 15 mai 2019 07:42, Simon Taddiken <si...@gmail.com> a
> écrit :
>
> > In my case, the pom of Y explicitly declares X with scope compile. The
> > dependency tree should look something like this:
> >
> > MyProject <- this is a standalone application, that will not be depended
> on
> > by s/o else
> > |- X:test
> > |- Y:compile
> >    \- X:compile
> >
> > So for MyProject, X is explicitly declared test but X also comes in
> > transitively via Y as compile. I do understand that declarations higher
> in
> > the tree are given precedence when it comes to dependency resolution but
> in
> > this case it simply breaks runtime code.
> > Of course I did not want to ship MyProject including X when I used X only
> > for tests. But now that Y also requires X I need MyProject to ship with X
> > in order to work correctly.
> >
> > I was just wondering whether there is some mechanism like a warning or a
> > flag to break the build in such cases, so that they do not go unnoticed.
> >
> >
> > Am Di., 14. Mai 2019 um 22:22 Uhr schrieb Jason Young <
> > jason.young@procentive.com>:
> >
> > > Did you declare that Y depends on X at all (via Y's pom.xml) or did it
> > > figure that out on its own (via transitive dependencies)?
> > >
> > > Test scope means it's for testing that one project, thus it's not
> > > transitive, e.g. I don't need JUnit for my tests just because
> > > SomeAwesomeProject uses JUnit for its tests, and I don't want to ship
> > JUnit
> > > in my project.
> > >
> > > On Tue, May 14, 2019 at 2:40 PM Simon Taddiken <
> simon.taddiken@gmail.com
> > >
> > > wrote:
> > >
> > > > Hi everyone,
> > > >
> > > > I've encountered the following behavior and I'm not quite sure
> whether
> > it
> > > > is desirable.
> > > > In my project, I have declared a dependency *X* with scope *test*. I
> > then
> > > > updated the version of a 3rd party dependency *Y*. In its new
> version,
> > > *Y*
> > > > suddenly requires the aforementioned dependency *X* as a *compile
> > *scoped
> > > > dependency.
> > > >
> > > > In this scenario, maven resolves the scope of *X* to be *test,
> > *although
> > > it
> > > > is now required by *Y* during runtime, causing
> ClassNotFoundExceptions.
> > > By
> > > > the very nature of this behavior those mistakes can't even be
> detected
> > in
> > > > unit tests because *X* is available on test classpath.
> > > >
> > > > I have just found out that this behavior seems to be intended (
> > > >
> > > >
> > >
> >
> https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Dependency_Scope
> > > > )
> > > > but I can't really come up with a rationale for this design. Managing
> > any
> > > > transitive compile scoped dependency down to test scope will almost
> > > > certainly cause ClassNotFoundExceptions during runtime.
> > > >
> > > > Thoughts?
> > > >
> > >
> >
>

Re: Maven adjusts transitive compile-scope dependency to test-scope

[Thomas Broyer <t....@gmail.com>]

There might be a rule in the enforcer plugin (I haven't checked). If one
really wanted to change the dependency scope, he could add an exclusion in
the Y dependency to break the transitivity.

Fwiw, whenever you change your dependencies, it's a good idea to run mvn
dependency:tree and look at the output (possibly diff'ing it with the
output prior to the dependency change).
Maven basically assumes that you take responsibility for the dependencies,
and it only takes the role of a helper resolving the whole tree.

Le mer. 15 mai 2019 07:42, Simon Taddiken <si...@gmail.com> a
écrit :

> In my case, the pom of Y explicitly declares X with scope compile. The
> dependency tree should look something like this:
>
> MyProject <- this is a standalone application, that will not be depended on
> by s/o else
> |- X:test
> |- Y:compile
>    \- X:compile
>
> So for MyProject, X is explicitly declared test but X also comes in
> transitively via Y as compile. I do understand that declarations higher in
> the tree are given precedence when it comes to dependency resolution but in
> this case it simply breaks runtime code.
> Of course I did not want to ship MyProject including X when I used X only
> for tests. But now that Y also requires X I need MyProject to ship with X
> in order to work correctly.
>
> I was just wondering whether there is some mechanism like a warning or a
> flag to break the build in such cases, so that they do not go unnoticed.
>
>
> Am Di., 14. Mai 2019 um 22:22 Uhr schrieb Jason Young <
> jason.young@procentive.com>:
>
> > Did you declare that Y depends on X at all (via Y's pom.xml) or did it
> > figure that out on its own (via transitive dependencies)?
> >
> > Test scope means it's for testing that one project, thus it's not
> > transitive, e.g. I don't need JUnit for my tests just because
> > SomeAwesomeProject uses JUnit for its tests, and I don't want to ship
> JUnit
> > in my project.
> >
> > On Tue, May 14, 2019 at 2:40 PM Simon Taddiken <simon.taddiken@gmail.com
> >
> > wrote:
> >
> > > Hi everyone,
> > >
> > > I've encountered the following behavior and I'm not quite sure whether
> it
> > > is desirable.
> > > In my project, I have declared a dependency *X* with scope *test*. I
> then
> > > updated the version of a 3rd party dependency *Y*. In its new version,
> > *Y*
> > > suddenly requires the aforementioned dependency *X* as a *compile
> *scoped
> > > dependency.
> > >
> > > In this scenario, maven resolves the scope of *X* to be *test,
> *although
> > it
> > > is now required by *Y* during runtime, causing ClassNotFoundExceptions.
> > By
> > > the very nature of this behavior those mistakes can't even be detected
> in
> > > unit tests because *X* is available on test classpath.
> > >
> > > I have just found out that this behavior seems to be intended (
> > >
> > >
> >
> https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Dependency_Scope
> > > )
> > > but I can't really come up with a rationale for this design. Managing
> any
> > > transitive compile scoped dependency down to test scope will almost
> > > certainly cause ClassNotFoundExceptions during runtime.
> > >
> > > Thoughts?
> > >
> >
>

Re: Maven adjusts transitive compile-scope dependency to test-scope

[Simon Taddiken <si...@gmail.com>]

In my case, the pom of Y explicitly declares X with scope compile. The
dependency tree should look something like this:

MyProject <- this is a standalone application, that will not be depended on
by s/o else
|- X:test
|- Y:compile
   \- X:compile

So for MyProject, X is explicitly declared test but X also comes in
transitively via Y as compile. I do understand that declarations higher in
the tree are given precedence when it comes to dependency resolution but in
this case it simply breaks runtime code.
Of course I did not want to ship MyProject including X when I used X only
for tests. But now that Y also requires X I need MyProject to ship with X
in order to work correctly.

I was just wondering whether there is some mechanism like a warning or a
flag to break the build in such cases, so that they do not go unnoticed.


Am Di., 14. Mai 2019 um 22:22 Uhr schrieb Jason Young <
jason.young@procentive.com>:

> Did you declare that Y depends on X at all (via Y's pom.xml) or did it
> figure that out on its own (via transitive dependencies)?
>
> Test scope means it's for testing that one project, thus it's not
> transitive, e.g. I don't need JUnit for my tests just because
> SomeAwesomeProject uses JUnit for its tests, and I don't want to ship JUnit
> in my project.
>
> On Tue, May 14, 2019 at 2:40 PM Simon Taddiken <si...@gmail.com>
> wrote:
>
> > Hi everyone,
> >
> > I've encountered the following behavior and I'm not quite sure whether it
> > is desirable.
> > In my project, I have declared a dependency *X* with scope *test*. I then
> > updated the version of a 3rd party dependency *Y*. In its new version,
> *Y*
> > suddenly requires the aforementioned dependency *X* as a *compile *scoped
> > dependency.
> >
> > In this scenario, maven resolves the scope of *X* to be *test, *although
> it
> > is now required by *Y* during runtime, causing ClassNotFoundExceptions.
> By
> > the very nature of this behavior those mistakes can't even be detected in
> > unit tests because *X* is available on test classpath.
> >
> > I have just found out that this behavior seems to be intended (
> >
> >
> https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Dependency_Scope
> > )
> > but I can't really come up with a rationale for this design. Managing any
> > transitive compile scoped dependency down to test scope will almost
> > certainly cause ClassNotFoundExceptions during runtime.
> >
> > Thoughts?
> >
>

Re: Maven adjusts transitive compile-scope dependency to test-scope

[Jason Young <ja...@procentive.com>]

Did you declare that Y depends on X at all (via Y's pom.xml) or did it
figure that out on its own (via transitive dependencies)?

Test scope means it's for testing that one project, thus it's not
transitive, e.g. I don't need JUnit for my tests just because
SomeAwesomeProject uses JUnit for its tests, and I don't want to ship JUnit
in my project.

On Tue, May 14, 2019 at 2:40 PM Simon Taddiken <si...@gmail.com>
wrote:

> Hi everyone,
>
> I've encountered the following behavior and I'm not quite sure whether it
> is desirable.
> In my project, I have declared a dependency *X* with scope *test*. I then
> updated the version of a 3rd party dependency *Y*. In its new version, *Y*
> suddenly requires the aforementioned dependency *X* as a *compile *scoped
> dependency.
>
> In this scenario, maven resolves the scope of *X* to be *test, *although it
> is now required by *Y* during runtime, causing ClassNotFoundExceptions. By
> the very nature of this behavior those mistakes can't even be detected in
> unit tests because *X* is available on test classpath.
>
> I have just found out that this behavior seems to be intended (
>
> https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Dependency_Scope
> )
> but I can't really come up with a rationale for this design. Managing any
> transitive compile scoped dependency down to test scope will almost
> certainly cause ClassNotFoundExceptions during runtime.
>
> Thoughts?
>