You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@maven.apache.org by lamyaa <gi...@git.apache.org> on 2015/09/01 01:24:13 UTC
[GitHub] maven-surefire pull request: Option to rerun failures at the end o...
GitHub user lamyaa opened a pull request:
https://github.com/apache/maven-surefire/pull/102
Option to rerun failures at the end of test execution
This PR introduces an option to rerun failing tests at the end of a test suite execution.
The current option (``rerunFailingTestsCount``) reruns the failing tests immediately after failure. There are several scenarios in which developers may want the failing tests to be rerun later. For example, consider a flaky test that tries to access a resource over a network. The test may fail whenever the network is down and rerunning it a bit later may be more helpful than rerunning it immediately (the network may recover later while it remains down immediately).
Note that the two options can be combined. For example, if a developer wants to rerun each failing test at most twice, the developer could choose to rerun the failing test once immediately, and if it still fails, rerun it once more at the end of the test suite. This strategy can be much better than rerunning a failing test twice immediately.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/lamyaa/maven-surefire rerun_failures_at_end
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/maven-surefire/pull/102.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #102
----
commit 967e3ef29d114752185da97aebbabaccab69b248
Author: lamyaa <el...@illinois.edu>
Date: 2015-08-30T03:40:21Z
Option to rerun failures at the end of test execution
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
[GitHub] maven-surefire pull request: Option to rerun failures at the end o...
Posted by lamyaa <gi...@git.apache.org>.
Github user lamyaa commented on the pull request:
https://github.com/apache/maven-surefire/pull/102#issuecomment-137231366
Oh, I see. I never looked into junit47 provide in detail. I'll look into it and see how rerunFailingTestsAtEndCount can fit there.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
[GitHub] maven-surefire pull request: Option to rerun failures at the end o...
Posted by Tibor17 <gi...@git.apache.org>.
Github user Tibor17 commented on the pull request:
https://github.com/apache/maven-surefire/pull/102#issuecomment-137228645
The junit47 provider runs all reruns in the after phase. If you use this provider, would it help the users? We should mention few differences between providers in documentation.
We internally decided to setup junit47 provider as the default one instead of nowadays junit4 provider. This means the provider junit4 will be marked as deprecated in surefire 3.0.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
[GitHub] maven-surefire pull request: Option to rerun failures at the end o...
Posted by Tibor17 <gi...@git.apache.org>.
Github user Tibor17 commented on the pull request:
https://github.com/apache/maven-surefire/pull/102#issuecomment-136962698
Pls import code style to your IDE.
https://maven.apache.org/developers/conventions/code.html#IntelliJ_IDEA_4.5
For instance in IntelliJ IDEA you import the XML code style and press CTRL+ALT+L to reformat the code.
Did you run the build (mvn install)?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
[GitHub] maven-surefire pull request: Option to rerun failures at the end o...
Posted by Tibor17 <gi...@git.apache.org>.
Github user Tibor17 commented on the pull request:
https://github.com/apache/maven-surefire/pull/102#issuecomment-143954421
We can try with junit4 of course.
On Tue, Sep 29, 2015 at 12:59 AM, lamyaa <no...@github.com> wrote:
> @Tibor17 <https://github.com/Tibor17> I just tried junit47 for the first
> time. Indeed, it seems to already run test at the end like I was attempting
> to do in this PR with junit4. Do you think it is not worth it to do this
> for junit4 if it will be deprecated in 3.0?
>
> —
> Reply to this email directly or view it on GitHub
> <https://github.com/apache/maven-surefire/pull/102#issuecomment-143896973>
> .
>
--
Cheers
Tibor
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
[GitHub] maven-surefire pull request: Option to rerun failures at the end o...
Posted by Tibor17 <gi...@git.apache.org>.
Github user Tibor17 commented on the pull request:
https://github.com/apache/maven-surefire/pull/102#issuecomment-136964837
Why you have `rerunAtEnd( reporterFactory, result.getFailures() );
` after `runNotifier.fireTestRunFinished( result );`?
I am not sure if test report would be updated after test run has finished.
What was intended?
How to configure `rerunFailingTestsAtEndCount` in order to reach all reruns in the after phase?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
[GitHub] maven-surefire pull request: Option to rerun failures at the end o...
Posted by Tibor17 <gi...@git.apache.org>.
Github user Tibor17 commented on the pull request:
https://github.com/apache/maven-surefire/pull/102#issuecomment-142747818
Hi @lamyaa
What should we do with this PR?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
[GitHub] maven-surefire pull request: Option to rerun failures at the end o...
Posted by lamyaa <gi...@git.apache.org>.
Github user lamyaa commented on the pull request:
https://github.com/apache/maven-surefire/pull/102#issuecomment-143896973
@Tibor17 I just tried junit47 for the first time. Indeed, it seems to already run test at the end like I was attempting to do in this PR with junit4. Do you think it is not worth it to do this for junit4 if it will be deprecated in 3.0?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
[GitHub] maven-surefire pull request: Option to rerun failures at the end o...
Posted by lamyaa <gi...@git.apache.org>.
Github user lamyaa commented on the pull request:
https://github.com/apache/maven-surefire/pull/102#issuecomment-142768756
@Tibor17 I haven't had a chance to look at junit47 provider yet. I'm hoping to get to it (as well as surefire-1144) this weekend.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
[GitHub] maven-surefire pull request: Option to rerun failures at the end o...
Posted by lamyaa <gi...@git.apache.org>.
Github user lamyaa commented on the pull request:
https://github.com/apache/maven-surefire/pull/102#issuecomment-137207940
> Did you run the build (mvn install)?
Yes, I did run mvn install and made sure that the checkstyle plugin passes. Are you maybe just seeing some weird line breaks in the github diff view because of the line lengths?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
[GitHub] maven-surefire pull request: Option to rerun failures at the end o...
Posted by lamyaa <gi...@git.apache.org>.
Github user lamyaa commented on the pull request:
https://github.com/apache/maven-surefire/pull/102#issuecomment-137210814
> I am not sure if test report would be updated after test run has finished.
What was intended?
From my understanding, rerunAtEnd again calls fireTestRunStarted and fireTestRunFinished. Perhaps, that's not needed if rerunAtEnd is called before the initial fireTestRunFinished, I can look into that.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org