You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@maven.apache.org by ol...@apache.org on 2026/10/14 01:04:22 UTC
(maven-surefire) 02/03: Document the runOrder changes in 3.6.x (#3510)
This is an automated email from the ASF dual-hosted git repository.
olamy pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/maven-surefire.git
commit eccd1535c9078cecd07944a6d667b81b954f7623
Author: Olivier Lamy <ol...@apache.org>
AuthorDate: Sun Oct 11 21:56:22 2026 +1000
Document the runOrder changes in 3.6.x (#3510)
Signed-off-by: Olivier Lamy <ol...@apache.org>
---
.../maven/plugin/failsafe/IntegrationTestMojo.java | 10 ++++++
.../apache/maven/plugin/surefire/SurefireMojo.java | 10 ++++++
.../fork-options-and-parallel-execution.md.vm | 2 +-
.../src/site/markdown/index.md.vm | 2 +-
.../{whats-new-3-6-0.md => whats-new-3-6-x.md} | 42 +++++++++++++++++++++-
maven-surefire-plugin/src/site/site.xml | 2 +-
src/site/markdown/index.md | 4 +--
7 files changed, 66 insertions(+), 6 deletions(-)
diff --git a/maven-failsafe-plugin/src/main/java/org/apache/maven/plugin/failsafe/IntegrationTestMojo.java b/maven-failsafe-plugin/src/main/java/org/apache/maven/plugin/failsafe/IntegrationTestMojo.java
index bc41aaeee..ca0810487 100644
--- a/maven-failsafe-plugin/src/main/java/org/apache/maven/plugin/failsafe/IntegrationTestMojo.java
+++ b/maven-failsafe-plugin/src/main/java/org/apache/maven/plugin/failsafe/IntegrationTestMojo.java
@@ -354,6 +354,16 @@ public class IntegrationTestMojo extends AbstractSurefireMojo {
* should not be checked into version control. The "XXXXX" is the SHA1 checksum of the entire surefire
* configuration, so different configurations will have different statistics files, meaning if you change any
* configuration settings you will re-run once before new statistics data can be established.
+ * <br>
+ * <br>
+ * Since 3.6.0, {@code alphabetical}, {@code reversealphabetical} and {@code random} also sort the test methods
+ * inside each class for JUnit 5/6 tests. A class with {@code @TestMethodOrder} keeps its own order. To keep the
+ * JUnit default method order, add an empty {@code junit.jupiter.testmethod.order.default=} to
+ * {@code configurationParameters} in {@code properties}. It has no effect in {@code junit-platform.properties}
+ * or as a system property.
+ * <br>
+ * <br>
+ * {@code hourly} was removed in 3.6.0. Use {@code random} instead.
*
* @since 2.7
*/
diff --git a/maven-surefire-plugin/src/main/java/org/apache/maven/plugin/surefire/SurefireMojo.java b/maven-surefire-plugin/src/main/java/org/apache/maven/plugin/surefire/SurefireMojo.java
index 8e75d27d0..fb8abc2d2 100644
--- a/maven-surefire-plugin/src/main/java/org/apache/maven/plugin/surefire/SurefireMojo.java
+++ b/maven-surefire-plugin/src/main/java/org/apache/maven/plugin/surefire/SurefireMojo.java
@@ -318,6 +318,16 @@ public class SurefireMojo extends AbstractSurefireMojo implements SurefireReport
* should not be checked into version control. The "XXXXX" is the SHA1 checksum of the entire surefire
* configuration, so different configurations will have different statistics files, meaning if you change any
* configuration settings you will re-run once before new statistics data can be established.
+ * <br>
+ * <br>
+ * Since 3.6.0, {@code alphabetical}, {@code reversealphabetical} and {@code random} also sort the test methods
+ * inside each class for JUnit 5/6 tests. A class with {@code @TestMethodOrder} keeps its own order. To keep the
+ * JUnit default method order, add an empty {@code junit.jupiter.testmethod.order.default=} to
+ * {@code configurationParameters} in {@code properties}. It has no effect in {@code junit-platform.properties}
+ * or as a system property.
+ * <br>
+ * <br>
+ * {@code hourly} was removed in 3.6.0. Use {@code random} instead.
*
* @since 2.7
*/
diff --git a/maven-surefire-plugin/src/site/markdown/examples/fork-options-and-parallel-execution.md.vm b/maven-surefire-plugin/src/site/markdown/examples/fork-options-and-parallel-execution.md.vm
index 580b92cc6..01667b4e6 100644
--- a/maven-surefire-plugin/src/site/markdown/examples/fork-options-and-parallel-execution.md.vm
+++ b/maven-surefire-plugin/src/site/markdown/examples/fork-options-and-parallel-execution.md.vm
@@ -48,7 +48,7 @@ Basically, there are two ways in ${project.artifactId} to achieve parallel test
Everything else is ignored: the suite values of `parallel` (`suites`, `suitesAndClasses`, `suitesAndMethods`, `all`) and
the deprecated `perCoreThreadCount`, `useUnlimitedThreads`, `threadCountSuites`, `parallelOptimized`,
`parallelTestsTimeoutInSeconds` and `parallelTestsTimeoutForcedInSeconds`. The rest of this section describes how
-the JUnit 4.7 provider of 3.5.x used them. See [What's New in 3.6.0](../whats-new-3-6-0.html#Parameter_Changes)
+the JUnit 4.7 provider of 3.5.x used them. See [What's New in 3.6.x](../whats-new-3-6-x.html#Parameter_Changes)
for details.
The most obvious one is by using the `parallel` parameter. The possible values depend on the test provider used. For JUnit 4.7 and onwards, this may be `methods`, `classes`, `both`, `suites`, `suitesAndClasses`, `suitesAndMethods`, `classesAndMethods` or `all`. As a prerequisite in JUnit tests, the JUnit runner should extend `org.junit.runners.ParentRunner`. If no runner is specified through the annotation `@org.junit.runner.RunWith`, the prerequisite is accomplished.
diff --git a/maven-surefire-plugin/src/site/markdown/index.md.vm b/maven-surefire-plugin/src/site/markdown/index.md.vm
index f7143f0e8..7c77e5028 100644
--- a/maven-surefire-plugin/src/site/markdown/index.md.vm
+++ b/maven-surefire-plugin/src/site/markdown/index.md.vm
@@ -112,4 +112,4 @@ The following examples show how to use the ${thisPlugin} Plugin in more advanced
- [Using Console Logs](./examples/logging.html)
- [Shutdown of Forked JVM](./examples/shutdown.html)
- [Run tests with Java 9](./java9.html)
-- [What's New in 3.6.0](./whats-new-3-6-0.html)
+- [What's New in 3.6.x](./whats-new-3-6-x.html)
diff --git a/maven-surefire-plugin/src/site/markdown/whats-new-3-6-0.md b/maven-surefire-plugin/src/site/markdown/whats-new-3-6-x.md
similarity index 82%
rename from maven-surefire-plugin/src/site/markdown/whats-new-3-6-0.md
rename to maven-surefire-plugin/src/site/markdown/whats-new-3-6-x.md
index 994796f11..42a19bb50 100644
--- a/maven-surefire-plugin/src/site/markdown/whats-new-3-6-0.md
+++ b/maven-surefire-plugin/src/site/markdown/whats-new-3-6-x.md
@@ -17,7 +17,7 @@ specific language governing permissions and limitations
under the License.
-->
-# What's New in Surefire 3.6.0
+# What's New in Surefire 3.6.x
## Overview
@@ -95,6 +95,7 @@ These apply to both `surefire:test` and `failsafe:integration-test` unless noted
| `parallelTestsTimeoutInSeconds`, `parallelTestsTimeoutForcedInSeconds` | Deprecated and ignored. | Use `forkedProcessTimeoutInSeconds`. |
| `objectFactory` | Deprecated and ignored. The TestNG engine has no such option. | Remove it. |
| `parallel` | Only some values still work, see below. | |
+| `runOrder` | Now also sorts JUnit 5/6 test methods. `hourly` is gone. | See [Run Order](#Run_Order) below. |
| `encoding` (`failsafe:verify` only) | Removed in 3.5.5. It had been deprecated since 2.20.1 and did nothing. | Remove it. |
Since 3.6.1 you get a warning when one of the ignored parameters is set. A legacy provider added as a plugin
@@ -110,6 +111,45 @@ Which `parallel` values still work:
- **JUnit 5/6**: `parallel` isn't used. Use the Jupiter `junit.jupiter.execution.parallel.*` settings instead,
see [Using JUnit 5 Platform](examples/junit-platform.html).
+### Run Order
+
+`alphabetical`, `reversealphabetical` and `random` now sort the test methods inside each class too, not only the
+classes. This only applies to JUnit 5/6 tests. JUnit 4 and TestNG methods run in the same order as before.
+
+| `runOrder` | Methods run |
+|------------|-------------|
+| `alphabetical` | by name |
+| `reversealphabetical` | by name, reversed |
+| `random` | in random order, using `runOrderRandomSeed` |
+
+A test that only passed because its methods ran in a certain order can now fail. This often happens with
+`@TestInstance(PER_CLASS)` classes that share state between methods.
+
+To keep the JUnit default method order, either:
+
+- add `@TestMethodOrder` to the class. Surefire never overrides it. If your methods depend on each other, this is
+ the best fix: use `@TestMethodOrder(MethodOrderer.OrderAnnotation.class)` and put `@Order` on each method.
+- or clear `junit.jupiter.testmethod.order.default`, the parameter Surefire sets, in `configurationParameters`.
+ Classes are still sorted by `runOrder`.
+
+```xml
+<configuration>
+ <runOrder>alphabetical</runOrder>
+ <properties>
+ <configurationParameters>
+ junit.jupiter.testmethod.order.default=
+ </configurationParameters>
+ </properties>
+</configuration>
+```
+
+With JUnit 6 you can also set it to `org.junit.jupiter.api.MethodOrderer$Default`. Setting it in
+`junit-platform.properties` or as a system property doesn't work, the value from Surefire wins.
+
+`hourly` was removed, and the build now fails with `There's no RunOrder with the name hourly`. It ran the tests
+alphabetically during even hours and in reverse during odd hours. Use `random` instead: the seed is printed in the
+build log, so you can replay a failing order with `runOrderRandomSeed`.
+
## Migration Guide
### JUnit 5/6 Users
diff --git a/maven-surefire-plugin/src/site/site.xml b/maven-surefire-plugin/src/site/site.xml
index 186151709..4c44032a3 100644
--- a/maven-surefire-plugin/src/site/site.xml
+++ b/maven-surefire-plugin/src/site/site.xml
@@ -28,7 +28,7 @@
<body>
<menu name="Overview">
<item name="Introduction" href="index.html"/>
- <item name="What's New in 3.6.0" href="whats-new-3-6-0.html"/>
+ <item name="What's New in 3.6.x" href="whats-new-3-6-x.html"/>
<item name="Plugin Documentation" href="plugin-info.html"/>
<item name="Usage" href="usage.html"/>
<item name="API" href="api.html"/>
diff --git a/src/site/markdown/index.md b/src/site/markdown/index.md
index ad69459fe..ca17cfac7 100644
--- a/src/site/markdown/index.md
+++ b/src/site/markdown/index.md
@@ -52,13 +52,13 @@ for the full list of resolved issues.
### 3.6.0-M1
The big one: every test framework now runs through a single, unified
-[JUnit Platform provider](maven-surefire-plugin/whats-new-3-6-0.html). JUnit 5 runs natively,
+[JUnit Platform provider](maven-surefire-plugin/whats-new-3-6-x.html). JUnit 5 runs natively,
and JUnit 4 and TestNG run through it too — giving you more consistent behaviour and far less
to configure. JUnit 5 tests can also run in parallel, and the JUnit 5 XML reports are tidier.
If you're upgrading, note that 3.6.0-M1 raises the minimum versions (JUnit 4.12+, TestNG
6.14.3+) and drops the old per-framework providers. The
-[What's New in 3.6.0](maven-surefire-plugin/whats-new-3-6-0.html) guide walks you through it,
+[What's New in 3.6.x](maven-surefire-plugin/whats-new-3-6-x.html) guide walks you through it,
and the [3.6.0-M1 release notes](https://github.com/apache/maven-surefire/releases/tag/surefire-3.6.0-M1)
list everything.