You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@maven.apache.org by "Hudson (JIRA)" <ji...@apache.org> on 2017/09/11 02:12:02 UTC
[jira] [Commented] (SUREFIRE-1264) Some tests can be lost when
running in parallel with parameterized tests
[ https://issues.apache.org/jira/browse/SUREFIRE-1264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16160555#comment-16160555 ]
Hudson commented on SUREFIRE-1264:
----------------------------------
SUCCESS: Integrated in Jenkins build maven-surefire #1757 (See [https://builds.apache.org/job/maven-surefire/1757/])
[SUREFIRE-1264] Some tests can be lost when running in parallel with (tibor17: [http://git-wip-us.apache.org/repos/asf/?p=maven-surefire.git&a=commit&h=c8724dd2f992625c355e4817a6b7995d3e88fdb4])
* (add) surefire-integration-tests/src/test/java/org/apache/maven/surefire/its/jiras/Surefire1264IT.java
* (edit) surefire-providers/common-junit4/src/main/java/org/apache/maven/surefire/junit4/MockReporter.java
* (edit) surefire-providers/surefire-junit47/src/main/java/org/apache/maven/surefire/junitcore/JUnitCoreRunListener.java
* (add) surefire-integration-tests/src/test/resources/surefire-1264/src/test/java/com/appnexus/viewability/core/surefireJunitTests/ATest.java
* (edit) surefire-providers/surefire-junit47/src/main/java/org/apache/maven/surefire/junitcore/TestSet.java
* (edit) surefire-providers/surefire-junit47/src/main/java/org/apache/maven/surefire/junitcore/ConcurrentRunListener.java
* (add) surefire-integration-tests/src/test/resources/surefire-1264/pom.xml
* (add) surefire-integration-tests/src/test/resources/surefire-1264/src/test/java/com/appnexus/viewability/core/surefireJunitTests/CTest.java
* (add) surefire-integration-tests/src/test/resources/surefire-1264/src/test/java/com/appnexus/viewability/core/surefireJunitTests/DTest.java
* (add) surefire-integration-tests/src/test/resources/surefire-1264/src/test/java/com/appnexus/viewability/core/surefireJunitTests/BaseTest.java
* (add) surefire-integration-tests/src/test/resources/surefire-1264/src/test/java/com/appnexus/viewability/core/surefireJunitTests/BTest.java
* (edit) surefire-providers/surefire-junit47/src/test/java/org/apache/maven/surefire/junitcore/JUnitCoreRunListenerTest.java
* (edit) surefire-api/src/main/java/org/apache/maven/surefire/util/internal/TestClassMethodNameUtils.java
* (add) surefire-integration-tests/src/test/resources/surefire-621-testCounting-junit3-in-parallel/src/it/java/mho/JUnit4AdapterSuiteTest.java
* (edit) surefire-integration-tests/src/test/java/org/apache/maven/surefire/its/jiras/Surefire621TestCountingJunit3InParallelIT.java
* (edit) surefire-providers/surefire-junit47/src/test/java/org/apache/maven/surefire/junitcore/TestMethodTest.java
* (edit) surefire-providers/common-junit4/src/main/java/org/apache/maven/surefire/common/junit4/JUnit4RunListener.java
* (edit) surefire-integration-tests/src/test/resources/surefire-621-testCounting-junit3-in-parallel/pom.xml
> Some tests can be lost when running in parallel with parameterized tests
> ------------------------------------------------------------------------
>
> Key: SUREFIRE-1264
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1264
> Project: Maven Surefire
> Issue Type: Bug
> Components: Junit 4.7+ (parallel) support
> Affects Versions: 2.19.1
> Environment: Maven 3.3.9
> Java 1.8.0_45 (Oracle)
> System: OS X
> Reproduced on Linux too, with both OpenJDK 7 and Java 7 from Oracle.
> Reporter: Jean-Luc Derrien
> Assignee: Tibor Digana
> Fix For: 2.20.1
>
> Attachments: failure-2.21-snapshot-SUREFIRE-1264.zip, pom.xml
>
> Time Spent: 144h
> Remaining Estimate: 24h
>
> Hello,
> It appears some tests can be lost when using the parallel mode with parameterized tests. Here is a [small project|https://github.com/jderrien/surefire-junit-tests/tree/simple-1] I've used to try to reproduce the issue we have seen.
> This is not something that happens on every run, but it's quite frequent.
> With this loop, the problem should appear in less than 2 minutes, maybe on the first run when (un)lucky:
> {code}
> time while true; do mvn clean test > last.log ; tail -25 last.log ; if [ "$(grep -c 'Tests run: 12' last.log)" == "0" ]; then break; fi ; done
> {code}
> Normal run:
> {noformat}
> -------------------------------------------------------
> T E S T S
> -------------------------------------------------------
> Running [p2]
> com.appnexus.viewability.core.surefireJunitTests.ATest.methodA1[p2] - p2 - sleeptime = 1 => start
> com.appnexus.viewability.core.surefireJunitTests.ATest.methodA1[p2] - p2 - sleeptime = 1 => stop
> com.appnexus.viewability.core.surefireJunitTests.ATest.methodA1[p1] - p1 - sleeptime = 2 => start
> com.appnexus.viewability.core.surefireJunitTests.ATest.methodA1[p1] - p1 - sleeptime = 2 => stop
> com.appnexus.viewability.core.surefireJunitTests.ATest.methodA1[p0] - p0 - sleeptime = 5 => start
> com.appnexus.viewability.core.surefireJunitTests.ATest.methodA1[p0] - p0 - sleeptime = 5 => stop
> Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.006 sec - in [p2]
> Running [p2]
> Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.011 sec - in [p2]
> Results :
> Tests run: 12, Failures: 0, Errors: 0, Skipped: 0
> {noformat}
> When tests have been lost (here one test lost according to the output):
> {noformat}
> -------------------------------------------------------
> T E S T S
> -------------------------------------------------------
> Running [p2]
> com.appnexus.viewability.core.surefireJunitTests.ATest.methodA1[p0] - p0 - sleeptime = 5 => start
> com.appnexus.viewability.core.surefireJunitTests.ATest.methodA1[p2] - p2 - sleeptime = 1 => start
> com.appnexus.viewability.core.surefireJunitTests.ATest.methodA1[p1] - p1 - sleeptime = 2 => start
> Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.007 sec - in [p2]
> Running [p2]
> Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.012 sec - in [p2]
> Results :
> Tests run: 11, Failures: 0, Errors: 0, Skipped: 0
> {noformat}
> Note: there are 3 test classes and 18 tests on the [master branch|https://github.com/jderrien/surefire-junit-tests/tree/master] and it's even more frequent/easy to reproduce.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)