You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@maven.apache.org by "Elliotte Rusty Harold (Jira)" <ji...@apache.org> on 2020/04/13 15:41:00 UTC
[jira] [Commented] (SUREFIRE-1638) untilFailureLoopCount property
to run tests until failure
[ https://issues.apache.org/jira/browse/SUREFIRE-1638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17082420#comment-17082420 ]
Elliotte Rusty Harold commented on SUREFIRE-1638:
-------------------------------------------------
It's a useful idea, though the name could be much better. Perhaps retryCount?
> untilFailureLoopCount property to run tests until failure
> ---------------------------------------------------------
>
> Key: SUREFIRE-1638
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1638
> Project: Maven Surefire
> Issue Type: New Feature
> Reporter: Galder Zamarreño
> Priority: Major
>
> This is a proposal to add surefire/failsafe property called `untilFailureLoopCount`.
> I often find myself in the situation where there's a random test failure happening in CI, and have to run the one or several tests continuously until it fails, maybe running them with TRACE logging.
> Normally, I'd wrap the `mvn test...` call with a script that calls it in a loop, but this can be quite "slow" since each iteration is a new JVM launch...etc.
> The idea behind `untilFailureLoopCount` is that you've give it a positive number, say 1000, and the test(s) would run continuously until either a failure happens or the test is executed that many times.
> I've created [this randomly failing test](https://github.com/galderz/surefire-until-failure-loop-count/blob/master/src/test/java/com/acme/maven/surefire/untilfailure/UntilFailureLoopCountTest.java) and I've run it with `mvn -Dsurefire.untilFailureLoopCount=10 test` and the output would be something like:
> {code}
> [INFO] -------------------------------------------------------
> [INFO] T E S T S
> [INFO] -------------------------------------------------------
> [INFO] Running com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest
> [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.034 s - in com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest
> [INFO] Running com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest
> [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 s - in com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest
> [INFO] Running com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest
> [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 s - in com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest
> [INFO] Running com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest
> [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 s - in com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest
> [INFO] Running com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest
> [ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.001 s <<< FAILURE! - in com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest
> [ERROR] com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest.testEventualFailure Time elapsed: 0.001 s <<< FAILURE!
> java.lang.AssertionError: Just messing with your testing
> at com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest.testEventualFailure(UntilFailureLoopCountTest.java:16)
> {code}
> If maintainers are interested in this, I can complete the PR with tests, documentation and support for other providers.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)