You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@maven.apache.org by fmeili <fr...@itac.de> on 2011/01/18 18:03:17 UTC
surefire classpath problems with maven 3 parallel
Hi,
while migrating to maven 3 (a large multi module project with >200 maven
projects) I ran into a lot of surefire JUnit problems. The whole project
builds correct with maven 3 serial build (without -T option).
I've isolated problematic tests and for them I use the surefire
configuration <forkMode>always</forkMode>. This solved my System-property
based parts of my projects so I'm generally able to build/test in parallel.
But I still have some strange ClassNotFound exceptions in parallel mode for
my JUnit tests. There are two different problems:
a)
While the JUnit tests are running it can't find any of the org.junit classes
(e.g. After, Befor, TestCase,...) and complains with a ClassNotFound for
them. I think I use the right dependencies in my projects root pom,
specified in the <dependencyManagement> section:
<dependencyManagement>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<scope>test</scope>
<version>4.8.2</version>
</dependency>
</dependencies>
</dependencyManagement>
The errors looks like this:
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.3.2:testCompile
(default-testCompile) on project qm-server: Compilation failure: Compilation
failure:
[ERROR]
/Daten/projects/qq/server/src/test/java/aaa/bbb/TestXYBean.java:[11,23]
package org.junit does not exist
[ERROR]
/Daten/projects/q/server/src/test/java/aaa/bbb//TestXYConfigBean.java:[15,3]
cannot find symbol
[ERROR] symbol : class Test
...
b)
The other problem occures for projects which doesn't have test classes until
yet (no JUnit tests in the src folder). But the surefire plugin try to
execute this (none existing tests) and this results in the following error:
------------------------------------------------------- T E S T
S-------------------------------------------------------
Concurrency config is parallel='none', perCoreThreadCount=true,
threadCount=2, useUnlimitedThreads=false
java.lang.reflect.UndeclaredThrowableException
at $Proxy0.invoke(Unknown Source)
at
org.apache.maven.surefire.booter.SurefireStarter.invokeProvider(SurefireStarter.java:145)
at
org.apache.maven.surefire.booter.SurefireStarter.runSuitesInProcess(SurefireStarter.java:87)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:69)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.maven.surefire.booter.ProviderFactory$ClassLoaderProxy.invoke(ProviderFactory.java:103)
... 4 more
Caused by: java.lang.NoClassDefFoundError:
org/junit/runner/notification/RunListener
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClassCond(ClassLoader.java:632)
at java.lang.ClassLoader.defineClass(ClassLoader.java:616)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
at
org.apache.maven.surefire.booter.IsolatedClassLoader.loadClass(IsolatedClassLoader.java:93)
at
org.apache.maven.surefire.junitcore.JUnitCoreProvider.invoke(JUnitCoreProvider.java:97)
... 9 more
Caused by: java.lang.ClassNotFoundException:
org.junit.runner.notification.RunListener
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
at
org.apache.maven.surefire.booter.IsolatedClassLoader.loadClass(IsolatedClassLoader.java:93)
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-surefire-plugin:2.7.1:test (default-test) on
project xy-server: There are test failures.
[ERROR]
[ERROR] Please refer to/Daten/projects/xy/server/target/surefire-reports for
the individual test results.
[ERROR] -> [Help 1]
This message indicates that a /surefile-reports directory exists, but it
doesn't exist because there are no tests. But the surefire plugin created a
directory /surefire with some tmp-files.
All my projects (also the projects which don't have unit tests until now)
inherit the JUnit dependency from my project root pom specified in the
<dependencies> section of my project root pom:
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
Playing around with the following surefire configuration changes the
behavior but not reproducable, sometimes some errors gone and a build later
the above errors appear on other projects and so on.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.7.1</version>
<configuration>
<forkMode>always</forkMode>
<useSystemClassLoader>true</useSystemClassLoader>
<useManifestOnlyJar>false</useManifestOnlyJar>
</configuration>
</plugin>
Til now I have not found a configuration where I can build massive parallel
with maven 3.
Has someone an idea to bring this to work? I'm using Maven 3.0.2.
Thanks in advance,
fmeili
--
View this message in context: http://maven.40175.n5.nabble.com/surefire-classpath-problems-with-maven-3-parallel-tp3346469p3346469.html
Sent from the Maven - Users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: surefire classpath problems with maven 3 parallel
Posted by fmeili <fr...@itac.de>.
With further analyzing I found that in case of the error (ClassNotFound) the
corresponding JAR file is really missing in the compiler class path while
the test classes are compiled. On some build runs, the junit jar was not in
the compiler classpath, on some other build runs, the mockito jar file was
not in the classpath. Sometimes both are missing. But sometimes a project
which faild on a build, runs ok when I try it again and it fails later on a
different project - but always with the same classpath problems. I found
this problems only on compiling test-classes, never on non test classes.
In the meantime someone opened a bug at codehouse which seems to describe
the same (or nearly the same) problem:
http://jira.codehaus.org/browse/MNG-4996?focusedCommentId=253613#action_253613
Thanks for your help,
Frank
--
View this message in context: http://maven.40175.n5.nabble.com/surefire-classpath-problems-with-maven-3-parallel-tp3346469p3364711.html
Sent from the Maven - Users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: surefire classpath problems with maven 3 parallel
Posted by Kristian Rosenvold <kr...@gmail.com>.
ma., 24.01.2011 kl. 00.19 -0800, skrev fmeili:
> Do you have any other idea how I may isolate the problem?
>
TestNG dependencies can also cause the same behaviour as I described in
my original mail (wrt junit3).
If that fails, there is no other way than to do as I said:
>Surefire stores the fork-configuration for the test run in the
>target/surefire folder, and if you "grep" for the correct files
>you should also be able to compare the values used for a
>successful/failing run. This would be really valuable.
>(/me kicks myself in the leg for not making those property files
>alphabetical for the 100th time. Create an issue if you feel the
same ;)
If you manage to isolate two different files that
give different results (for the same test run), I would recommend you
create a SUREFIRE issue and attach the two files. If you're using
forkMode=once it should be fairly simple, it might be a little more work
for forkMode=always ;) It should be as simple as tucking away the
properties file for a build that works, run until it fails and
find the (nev version) of the same property file.
> b)
> Very interesting, If I don't specify the 4.7 provider explicitly, this
> problem has gone
When you specify the 4.7 provider as a dependency, that overrides any
automatic detection.
As long as you activate through setting the "parallel" attribute
(instead of using the new forced provider mode in 2.7), you will be
using the built-in surefire provider detection. While I suspect this
will give you a different provider (and might seem to work), it may also
have the unintended side effect of not running junit4 tests :(
Kristian
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: surefire classpath problems with maven 3 parallel
Posted by fmeili <fr...@itac.de>.
Hi Kristian,
thanks for your quick answer.
a)
I've tried a lot of things without success. As you told me, I was looking
for hidden JUnit 3 dependency, but have not found any. To check this, I
built with -X and scan the stdout for junit3. There are some junit3 test
dependencies in some standard maven plugins but I think this does not hurt
my build (because I don't build the maven plugins I only use them).
You've asked about if it fails, does it always fail on the same set of
modules. Yes, it fails always on the same modules but depending of the build
(pseudo random) order a module failed on the first build but may work on the
second build and it fails on a 3rd module. On some builds the junit classes
are not found, on some other builds the mockito classes are not found. But
for each module it fails always with the same error if it fails.
I've tried to specify the dependencies for junit/mockito directly in the
modules pom, even I've specified it in the projects root pom. Without
success.
Additionally I've played around with older versions of the surefire plugin
(beginning from version 2.6) and with different settings for the surefire
configuration. I've also tried to use older versions of junit (beginning
from Junit 4.4). But nothing helps.
Sometimes my build works with -T 2 (each 3rd try it works). Without -T
option it builds always successfully. With -T 3 it never builds with
success. (My project is a multi module project with 170 projects).
Do you have any other idea how I may isolate the problem?
b)
Very interesting, If I don't specify the 4.7 provider explicitly, this
problem has gone.
Thanks,
Frank
--
View this message in context: http://maven.40175.n5.nabble.com/surefire-classpath-problems-with-maven-3-parallel-tp3346469p3354341.html
Sent from the Maven - Users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: surefire classpath problems with maven 3 parallel
Posted by Kristian Rosenvold <kr...@gmail.com>.
Den 18.01.2011 18:03, skrev fmeili:
a)
The symptoms you are encountering seem to indicate that you're getting
testng or junit3 in somewhere on your
classpath. As to why this is happening, we'll have to try some detective
work:
The most significant difference between serial and parallel maven
run-mode is that in serial, the entire
run-order of the reactor is defined up-front. In parallel, it isn't.
Technically parallel mode knows a range of *possible* run-orders that
are described by the dependencies in the poms of your individual
modules. So from build to build the run-order of your modules can
change somewhat. If module number 36 for some reason decides to spend
twice as much time as usual, it could change the
build order of the remaining build more or less completely (although
it'll normally not be *that* bad).
So mvn dependency:analyze is your friend here. Especially the "used,
undeclared" dependencies might be able to introduce non-determinisms
into your parallel build.
A few more questions to ask:
- Although it fails intermittently, does it always fail on the same set
of modules ?
- Does supressing transitive junit3/testng inclusions help ?
Surefire stores the fork-configuration for the test run in the
target/surefire folder, and if you "grep" for the correct files
you should also be able to compare the values used for a
successful/failing run. This would be really valuable.
(/me kicks myself in the leg for not making those property files
alphabetical for the 100th time. Create an issue if you feel the same ;)
b)
This may actually be caused by somehow picking up testng/junit3 (ref
problem A), especially if you're forcing surefire to use
the 4.7 provider. It's probably some behavioural change related to
http://jira.codehaus.org/browse/SUREFIRE-663,
although I am not entirely sure of what would cause it.
Kristian
> Hi,
>
> while migrating to maven 3 (a large multi module project with>200 maven
> projects) I ran into a lot of surefire JUnit problems. The whole project
> builds correct with maven 3 serial build (without -T option).
>
> I've isolated problematic tests and for them I use the surefire
> configuration<forkMode>always</forkMode>. This solved my System-property
> based parts of my projects so I'm generally able to build/test in parallel.
> But I still have some strange ClassNotFound exceptions in parallel mode for
> my JUnit tests. There are two different problems:
>
> a)
> While the JUnit tests are running it can't find any of the org.junit classes
> (e.g. After, Befor, TestCase,...) and complains with a ClassNotFound for
> them. I think I use the right dependencies in my projects root pom,
> specified in the<dependencyManagement> section:
>
> <dependencyManagement>
> <dependencies>
> <dependency>
> <groupId>junit</groupId>
> <artifactId>junit</artifactId>
> <scope>test</scope>
> <version>4.8.2</version>
> </dependency>
> </dependencies>
> </dependencyManagement>
>
> The errors looks like this:
>
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:2.3.2:testCompile
> (default-testCompile) on project qm-server: Compilation failure: Compilation
> failure:
> [ERROR]
> /Daten/projects/qq/server/src/test/java/aaa/bbb/TestXYBean.java:[11,23]
> package org.junit does not exist
> [ERROR]
> /Daten/projects/q/server/src/test/java/aaa/bbb//TestXYConfigBean.java:[15,3]
> cannot find symbol
> [ERROR] symbol : class Test
> ...
>
> b)
> The other problem occures for projects which doesn't have test classes until
> yet (no JUnit tests in the src folder). But the surefire plugin try to
> execute this (none existing tests) and this results in the following error:
>
> ------------------------------------------------------- T E S T
> S-------------------------------------------------------
> Concurrency config is parallel='none', perCoreThreadCount=true,
> threadCount=2, useUnlimitedThreads=false
> java.lang.reflect.UndeclaredThrowableException
> at $Proxy0.invoke(Unknown Source)
> at
> org.apache.maven.surefire.booter.SurefireStarter.invokeProvider(SurefireStarter.java:145)
> at
> org.apache.maven.surefire.booter.SurefireStarter.runSuitesInProcess(SurefireStarter.java:87)
> at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:69)
>
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.maven.surefire.booter.ProviderFactory$ClassLoaderProxy.invoke(ProviderFactory.java:103)
> ... 4 more
>
> Caused by: java.lang.NoClassDefFoundError:
> org/junit/runner/notification/RunListener
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClassCond(ClassLoader.java:632)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:616)
> at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
> at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
> at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> at
> org.apache.maven.surefire.booter.IsolatedClassLoader.loadClass(IsolatedClassLoader.java:93)
> at
> org.apache.maven.surefire.junitcore.JUnitCoreProvider.invoke(JUnitCoreProvider.java:97)
> ... 9 more
>
> Caused by: java.lang.ClassNotFoundException:
> org.junit.runner.notification.RunListener
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> at
> org.apache.maven.surefire.booter.IsolatedClassLoader.loadClass(IsolatedClassLoader.java:93)
>
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-surefire-plugin:2.7.1:test (default-test) on
> project xy-server: There are test failures.
> [ERROR]
> [ERROR] Please refer to/Daten/projects/xy/server/target/surefire-reports for
> the individual test results.
> [ERROR] -> [Help 1]
>
> This message indicates that a /surefile-reports directory exists, but it
> doesn't exist because there are no tests. But the surefire plugin created a
> directory /surefire with some tmp-files.
>
> All my projects (also the projects which don't have unit tests until now)
> inherit the JUnit dependency from my project root pom specified in the
> <dependencies> section of my project root pom:
>
> <dependencies>
> <dependency>
> <groupId>junit</groupId>
> <artifactId>junit</artifactId>
> <scope>test</scope>
> </dependency>
> </dependencies>
>
>
> Playing around with the following surefire configuration changes the
> behavior but not reproducable, sometimes some errors gone and a build later
> the above errors appear on other projects and so on.
>
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-surefire-plugin</artifactId>
> <version>2.7.1</version>
> <configuration>
> <forkMode>always</forkMode>
> <useSystemClassLoader>true</useSystemClassLoader>
> <useManifestOnlyJar>false</useManifestOnlyJar>
> </configuration>
> </plugin>
>
> Til now I have not found a configuration where I can build massive parallel
> with maven 3.
>
> Has someone an idea to bring this to work? I'm using Maven 3.0.2.
>
> Thanks in advance,
> fmeili
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org