You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@maven.apache.org by "olamy (via GitHub)" <gi...@apache.org> on 2026/09/11 01:24:02 UTC
[PR] Issue #3462 TestNG extension onStart/onFinish called too many times [maven-surefire]
olamy opened a new pull request, #3470:
URL: https://github.com/apache/maven-surefire/pull/3470
Signed-off-by: Olivier Lamy <ol...@apache.org>
Following this checklist to help us incorporate your
contribution quickly and easily:
- [ ] Each commit in the pull request should have a meaningful subject line and body.
- [ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
- [ ] Run `mvn clean install` to make sure basic checks pass. A more thorough check will
be performed on your pull request automatically.
- [ ] You have run the integration tests successfully (`mvn -Prun-its clean install`).
If your pull request is about ~20 lines of code you don't need to sign an
[Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf) if you are unsure
please ask on the developers list.
To make clear that you license your contribution under
the [Apache License Version 2.0, January 2004](http://www.apache.org/licenses/LICENSE-2.0)
you have to acknowledge this by using the following check-box.
- [ ] I hereby declare this contribution to be licenced under the [Apache License Version 2.0, January 2004](http://www.apache.org/licenses/LICENSE-2.0)
- [ ] In any other case, please file an [Apache Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Issue #3462 TestNG extension onStart/onFinish called too many times [maven-surefire]
Posted by "lprimak (via GitHub)" <gi...@apache.org>.
lprimak commented on PR #3470:
URL: https://github.com/apache/maven-surefire/pull/3470#issuecomment-6103317227
Confirmed. Only called 2 times with the PR in it's current state.
Maybe we wait for JUnit team to respond to the new issue?
I am good either way, this PR is better than the status quo, and we can improve later when/if JUnit team resolves the issue.
> Thanks for testing! I pushed a change so the class scan no longer asks the TestNG engine, which brings it down to 2. The remaining extra call is the testng-engine's dry run during discovery (source). The JUnit Platform is always discovered before execution, so Surefire can't avoid it. Have a squeeze and let me know.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Issue #3462 TestNG extension onStart/onFinish called too many times [maven-surefire]
Posted by "olamy (via GitHub)" <gi...@apache.org>.
olamy commented on PR #3470:
URL: https://github.com/apache/maven-surefire/pull/3470#issuecomment-6096156347
Thanks for testing!
In your case, 3 is currently expected: the testng-engine discovers tests by running TestNG in dry-run mode, which notifies ISuiteListeners, and the JUnit Platform always discovers before executing. (see comment here https://github.com/apache/maven-surefire/issues/3462#issuecomment-5627432284 and javadoc here https://github.com/junit-team/testng-engine/blob/78e75694529cca30bb741d4363c318ecaa2058d5/src/main/java/org/junit/support/testng/engine/TestNGTestEngine.java#L82-L89 )
So 1 isn't reachable from Surefire alone. Surefire still discovers once more than it needs to during the class scan, though, and if possible, I'll look at getting that down to 2.
In the meantime, your listener can skip the dry runs with if (Boolean.getBoolean("testng.mode.dryrun")) return; and you'll get exactly one call.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Issue #3462 TestNG extension onStart/onFinish called too many times [maven-surefire]
Posted by "olamy (via GitHub)" <gi...@apache.org>.
olamy commented on PR #3470:
URL: https://github.com/apache/maven-surefire/pull/3470#issuecomment-6103272407
Thanks for testing! I pushed a change so the class scan no longer asks the TestNG engine, which brings it down to 2. The remaining extra call is the testng-engine's dry run during discovery (source). The JUnit Platform is always discovered before execution, so Surefire can't avoid it.
Have a squeeze and let me know.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Issue #3462 TestNG extension onStart/onFinish called too many times [maven-surefire]
Posted by "lprimak (via GitHub)" <gi...@apache.org>.
lprimak commented on PR #3470:
URL: https://github.com/apache/maven-surefire/pull/3470#issuecomment-6101189240
Thanks for the explanation, and the `testng.mode.dryrun` workaround works for me. Still, listeners can't tell a discovery pass from a real suite without knowing that internal property, which is surprising. I opened an issue in testng-engine to discuss avoiding or signaling this: https://github.com/junit-team/testng-engine/issues/440
Meanwhile, 2 calls from Surefire (one discovery + one execution) is a reasonable interim state, and this PR is a clear improvement over 4.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Issue #3462 TestNG extension onStart/onFinish called too many times [maven-surefire]
Posted by "lprimak (via GitHub)" <gi...@apache.org>.
lprimak commented on PR #3470:
URL: https://github.com/apache/maven-surefire/pull/3470#issuecomment-5640128396
With this PR, I get the below output.
I would still say that's "too many times"
I think that the extension should only be called once, and it's getting called 3 times now (was 4 times before this fix)
```
[INFO] --- surefire:3.6.1-SNAPSHOT:test (default-test) @ maven-testng-reproducer ---
[INFO] Using auto detected provider org.apache.maven.surefire.junitplatform.JUnitPlatformProvider
[INFO] TestNG is present. Adding per default org.junit.support:testng-engine:1.1.0
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
MyTestNGExtension: onStart called for suite: Command line suite
MyTestNGExtension: onFinish called for suite: Command line suite
MyTestNGExtension: onStart called for suite: Command line suite
MyTestNGExtension: onFinish called for suite: Command line suite
[INFO] Running com.flowlogix.maven.jupiter.JupiterTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.016 s -- in com.flowlogix.maven.jupiter.JupiterTest
MyTestNGExtension: onStart called for suite: Command line suite
[INFO] Running com.flowlogix.maven.testng.NextGenTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.003 s -- in com.flowlogix.maven.testng.NextGenTest
MyTestNGExtension: onFinish called for suite: Command line suite
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Issue #3462 TestNG extension onStart/onFinish called too many times [maven-surefire]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #3470:
URL: https://github.com/apache/maven-surefire/pull/3470#issuecomment-6105639184
@olamy Please assign appropriate label to PR according to the type of change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Issue #3462 TestNG extension onStart/onFinish called too many times [maven-surefire]
Posted by "olamy (via GitHub)" <gi...@apache.org>.
olamy merged PR #3470:
URL: https://github.com/apache/maven-surefire/pull/3470
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org