You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@maven.apache.org by qingzhouluo <gi...@git.apache.org> on 2014/06/24 22:48:48 UTC
[GitHub] maven-surefire pull request: Add rerunFailingTests option for mave...
GitHub user qingzhouluo opened a pull request:
https://github.com/apache/maven-surefire/pull/40
Add rerunFailingTests option for maven surefire to rerun failing tests i...
Add rerunFailingTests option for maven surefire to rerun failing tests immediately after they fail. Each single test method now maintains a list of its runs. DefaultReporterFactory now contains a few maps between test and the list of all its runs. If a test passes in any of its reruns, the build will be marked as successful and the test will count as flake.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/qingzhouluo/maven-surefire master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/maven-surefire/pull/40.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #40
----
commit 37b711b4bc244b9947b6ee9d37b1163eb8217262
Author: Qingzhou Luo <qi...@google.com>
Date: 2014-06-24T20:44:44Z
Add rerunFailingTests option for maven surefire to rerun failing tests immediately after they fail. Each single test method now maintains a list of its runs. DefaultReporterFactory now contains a few maps between test and the list of all its runs. If a test passes in any of its reruns, the build will be marked as successful and the test will count as flake.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: [GitHub] maven-surefire pull request: Add rerunFailingTests
option for mave...
Posted by Paul Benedict <pb...@apache.org>.
Hi,
You sure created a big patch! I am sure there is more than one way to solve
this problem. Enhancing the plugin is one way, but is that really the right
way to universalize the functionality? I don't know. Maybe others will
agree with you. My first thoughts would be that the external process that
called Maven should read the test report (XML) and determine whether to
re-run.
Paul
Cheers,
Paul
On Tue, Jun 24, 2014 at 3:48 PM, qingzhouluo <gi...@git.apache.org> wrote:
> GitHub user qingzhouluo opened a pull request:
>
> https://github.com/apache/maven-surefire/pull/40
>
> Add rerunFailingTests option for maven surefire to rerun failing tests
> i...
>
> Add rerunFailingTests option for maven surefire to rerun failing tests
> immediately after they fail. Each single test method now maintains a list
> of its runs. DefaultReporterFactory now contains a few maps between test
> and the list of all its runs. If a test passes in any of its reruns, the
> build will be marked as successful and the test will count as flake.
>
> You can merge this pull request into a Git repository by running:
>
> $ git pull https://github.com/qingzhouluo/maven-surefire master
>
> Alternatively you can review and apply these changes as the patch at:
>
> https://github.com/apache/maven-surefire/pull/40.patch
>
> To close this pull request, make a commit to your master/trunk branch
> with (at least) the following in the commit message:
>
> This closes #40
>
> ----
> commit 37b711b4bc244b9947b6ee9d37b1163eb8217262
> Author: Qingzhou Luo <qi...@google.com>
> Date: 2014-06-24T20:44:44Z
>
> Add rerunFailingTests option for maven surefire to rerun failing tests
> immediately after they fail. Each single test method now maintains a list
> of its runs. DefaultReporterFactory now contains a few maps between test
> and the list of all its runs. If a test passes in any of its reruns, the
> build will be marked as successful and the test will count as flake.
>
> ----
>
>
> ---
> If your project is set up for it, you can reply to this email and have your
> reply appear on GitHub as well. If your project does not have this feature
> enabled and wishes so, or if the feature is enabled but not working, please
> contact infrastructure at infrastructure@apache.org or file a JIRA ticket
> with INFRA.
> ---
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
Re: [GitHub] maven-surefire pull request: Add rerunFailingTests option
for mave...
Posted by Karl Heinz Marbaise <kh...@gmx.de>.
Hi,
I have some thoughts about that...first as far as i can remember we had
a discussion about the same subject a longer time ago...
Furthermore by definition a unit test is just a unit which is very small
and should not have any relationships to anything outside (exception:
mocking)...
So if a unit test fails it fails....this can't getting better if we
simply repeat it...
So i think we are talking more about integration tests with some
relationship to sourounding systems....where i can understand the idea...
which brings me to the patch...so that patch should be more related to
the maven-failsafe-plugin than the maven-surefire-plugin......
After diving into that i have found a library on google code which
already supports that for JUnit with an appropriate annotation which
exactly does what you need (apart from the point that JUnit is unit test
framework and NOT an integration test framework like TestNG, but this is
a different story).
http://tempusfugitlibrary.org/
http://tempusfugitlibrary.org/documentation/junit/intermittent/
@RunWith(IntermittentTestRunner.class)
@Intermittent(repetition = 10)
public class IntermittentTestRunnerTest {
private static int testCounter = 0;
@Test
public void annotatedTest() {
testCounter++;
}
@Test
public void anotherAnnotatedTest() {
testCounter++;
}
@AfterClass
public static void assertAnnotatedTestRunsMultipleTimes() {
assertThat(testCounter, is(equalTo(20)));
}
}
So this makes it much more cleared that you integration tests will run
several times...
If you make a change on the level of maven-{surefire,failsafe}-plugin
you will change it for all tests which means you have to handle things
via includes/excludes which will make you pom very confusing...
From my point of view I don't see it to change surefire neither
failsafe into this direction...may be the other dev's have different
opinions.
Kind regards
Karl-Heinz Marbaise
On 6/24/14 10:48 PM, qingzhouluo wrote:
> GitHub user qingzhouluo opened a pull request:
>
> https://github.com/apache/maven-surefire/pull/40
>
> Add rerunFailingTests option for maven surefire to rerun failing tests i...
>
> Add rerunFailingTests option for maven surefire to rerun failing tests immediately after they fail. Each single test method now maintains a list of its runs. DefaultReporterFactory now contains a few maps between test and the list of all its runs. If a test passes in any of its reruns, the build will be marked as successful and the test will count as flake.
>
> You can merge this pull request into a Git repository by running:
>
> $ git pull https://github.com/qingzhouluo/maven-surefire master
>
> Alternatively you can review and apply these changes as the patch at:
>
> https://github.com/apache/maven-surefire/pull/40.patch
>
> To close this pull request, make a commit to your master/trunk branch
> with (at least) the following in the commit message:
>
> This closes #40
>
> ----
> commit 37b711b4bc244b9947b6ee9d37b1163eb8217262
> Author: Qingzhou Luo <qi...@google.com>
> Date: 2014-06-24T20:44:44Z
>
> Add rerunFailingTests option for maven surefire to rerun failing tests immediately after they fail. Each single test method now maintains a list of its runs. DefaultReporterFactory now contains a few maps between test and the list of all its runs. If a test passes in any of its reruns, the build will be marked as successful and the test will count as flake.
>
> ----
>
>
> ---
> If your project is set up for it, you can reply to this email and have your
> reply appear on GitHub as well. If your project does not have this feature
> enabled and wishes so, or if the feature is enabled but not working, please
> contact infrastructure at infrastructure@apache.org or file a JIRA ticket
> with INFRA.
> ---
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
[GitHub] maven-surefire pull request: Add rerunFailingTests option for mave...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/maven-surefire/pull/40
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org