You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@maven.apache.org by GitBox <gi...@apache.org> on 2022/06/07 09:13:18 UTC
[GitHub] [maven-surefire] br0nstein opened a new pull request, #545: [SUREFIRE-2095] Fork crash doesn't fail build with -Dmaven.test.failure.ignore=true when run with failsafe
br0nstein opened a new pull request, #545:
URL: https://github.com/apache/maven-surefire/pull/545
When `maven.test.failure.ignore` is enabled, a test that crashes the JVM (e.g. that causes an OOM, with `-XX:+CrashOnOutOfMemoryError`) currently does not cause a build failure and is not reported to the test summary XML in a way that CI tools like Jenkins treat as indicating test failures (to mark the job result unstable). This PR ensures that whether test failures are configured to be ignored or not, such a crash in the forked process by the failsafe plugin will always fail the build.
This PR updates the VerifyMojo to pass a SurefireBooterForkException (deserialized from the failureMessage output to the summary XML from the IntegrationTestMojo run prior) to reportExecution so the build is terminated with a MojoExecutionException the same way as it is when tests are run by the surefire plugin (AbstractSurefireMojo).
See 6e60b0389814d8361e453092d3b18f52c3e4bcb1 for the corresponding fix for the surefire plugin (SUREFIRE-1426), that this leverages.
Following this checklist to help us incorporate your
contribution quickly and easily:
- [x] Make sure there is a [JIRA issue](https://issues.apache.org/jira/browse/SUREFIRE) filed
for the change (usually before you start working on it). Trivial changes like typos do not
require a JIRA issue. Your pull request should address just this issue, without
pulling in other changes.
- [x] Each commit in the pull request should have a meaningful subject line and body.
- [x] Format the pull request title like `[SUREFIRE-XXX] - Fixes bug in ApproximateQuantiles`,
where you replace `SUREFIRE-XXX` with the appropriate JIRA issue. Best practice
is to use the JIRA issue title in the pull request title and in the first line of the
commit message.
- [x] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
- [x] Run `mvn clean install` to make sure basic checks pass. A more thorough check will
be performed on your pull request automatically.
- [x] You have run the integration tests successfully (`mvn -Prun-its clean install`).
If your pull request is about ~20 lines of code you don't need to sign an
[Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf) if you are unsure
please ask on the developers list.
To make clear that you license your contribution under
the [Apache License Version 2.0, January 2004](http://www.apache.org/licenses/LICENSE-2.0)
you have to acknowledge this by using the following check-box.
- [x] I hereby declare this contribution to be licenced under the [Apache License Version 2.0, January 2004](http://www.apache.org/licenses/LICENSE-2.0)
- [ ] In any other case, please file an [Apache Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [maven-surefire] Tibor17 commented on pull request #545: [SUREFIRE-2095] Fork crash doesn't fail build with -Dmaven.test.failure.ignore=true when run with failsafe
Posted by GitBox <gi...@apache.org>.
Tibor17 commented on PR #545:
URL: https://github.com/apache/maven-surefire/pull/545#issuecomment-1151747378
@br0nstein
Thx for the fix in the verifier mojo.
I admit the commit [6e60b03](https://github.com/apache/maven-surefire/commit/6e60b0389814d8361e453092d3b18f52c3e4bcb1) was not done so well. The situation with XML reports is even more complicated if you have `forkCount` greater than `1`.
Additionally, I think we touched the discussion with the non-zero exit where the exit on-jvm-startup is different situation from on-test-run-jvm.
Let's continue and discuss it on tomorrow.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [maven-surefire] olamy merged pull request #545: [SUREFIRE-2095] Fork crash doesn't fail build with -Dmaven.test.failure.ignore=true when run with failsafe
Posted by "olamy (via GitHub)" <gi...@apache.org>.
olamy merged PR #545:
URL: https://github.com/apache/maven-surefire/pull/545
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [maven-surefire] br0nstein commented on pull request #545: [SUREFIRE-2095] Fork crash doesn't fail build with -Dmaven.test.failure.ignore=true when run with failsafe
Posted by GitBox <gi...@apache.org>.
br0nstein commented on PR #545:
URL: https://github.com/apache/maven-surefire/pull/545#issuecomment-1150346301
@olamy I updated the PR description to add some clarification. I don't think this is really the best solution but I did it this way to align with what you and @Tibor17 agreed on for surefire-1426.
I think ideally for both surefire and failsafe the build succeeds (as documented for maven.test.failure.ignore), however somewhere in ForkStarter when we see the non-zero exit code we make sure to create the TEST-*.xml file. For the case of a crash starting the JVM, from debugging I see that `forkClient.hasTestsInProgress()` is false. So not exactly sure what to do there - is it totally wrong to use the `scanResult.classes` to identify which files to create? (That said it just has simple class names.) Since these tests weren't at fault though it might make sense to create a TEST file for the running module (using groupId+artifactId)?
However, for the case where a test was running that crashed the JVM, we do have the full class name in `forkClient.testsInProgress()`and could use that directly to create the TEST-*.xml file. I believe we do not need to populate the individual test cases in the XML - perhaps it isn't well documented but I see the Jenkins Junit plugin for one [checks for the presence of an error testSuite element](https://github.com/jenkinsci/junit-plugin/blob/junit-1.64/src/main/java/hudson/tasks/junit/SuiteResult.java#L241).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [maven-surefire] br0nstein commented on pull request #545: [SUREFIRE-2095] Fork crash doesn't fail build with -Dmaven.test.failure.ignore=true when run with failsafe
Posted by GitBox <gi...@apache.org>.
br0nstein commented on PR #545:
URL: https://github.com/apache/maven-surefire/pull/545#issuecomment-1226895900
@Tibor17 thanks please let me know how we can go from here to get this problem fixed. Should we try to update the test to actually crash the JVM at runtime rather than cause JVM startup to fail in case that were to be caught by failsafe differently? Not sure if there are any ripe unpatched JVM bugs that we could utilize, heh.
Want to double check that we do want to pursue this approach of causing the verify phase to fail rather than treat this like non-crashing failing tests with test.failure.ignore enabled, by writing out some test result XML that Jenkins/etc could pick up to treat as "unstable."
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org